AI generates video game levels and characters from text prompts

A generative AI model based on small datasets was able to create maps and 2D character models for video games on demand

By Chris Stokel-Walker

1 September 2023

“Duck” and “ghost” character sprites generated by an AI

Tim Merino et al

A simple generative AI tool can create video game maps, character models and emojis from a single-sentence prompt within milliseconds.

Julian Togelius and Timothy Merino at New York University and their colleagues designed the system as a way of understanding how simple an AI model can be while still proving useful.

“We tried as a starting point to figure out the most naive, simple approach we could do for textbook map generation,” says Merino. “It was surprisingly effective.”

Advertisement

The model was trained on databases of 882 game maps, 100 game sprites and 10,000 emojis, all of which were labelled with descriptions of what the images showed. “All of our datasets were fairly small, and that was by design,” says Merino.

The labelling avoided listing the specific names of characters, instead describing Mario as “a man with a moustache dressed in red”, for example. Alternative labels were also created to train the model using GPT-4, the large language model behind ChatGPT.