The Five-Dollar Model: Generating Game Maps and Sprites from Sentence Embeddings

Authors

  • Timothy Merino New York University - Tandon School of Engineering
  • Roman Negri New York University - Tandon School of Engineering
  • Dipika Rajesh Independent Researcher
  • M Charity New York University - Tandon School of Engineering
  • Julian Togelius New York University - Tandon School of Engineering

DOI:

https://doi.org/10.1609/aiide.v19i1.27506

Keywords:

Procedural Content Generation, Machine Learning, Text-to-image, Image Generation, Low Dimensional Data, Art Generation, Natural Language Understanding, Small Scale Learning

Abstract

The five-dollar model is a lightweight text-to-image generative architecture that generates low dimensional images or tile maps from an encoded text prompt. This model can successfully generate accurate and aesthetically pleasing content in low dimensional domains, with limited amounts of training data. Despite the small size of both the model and datasets, the generated images or maps are still able to maintain the encoded semantic meaning of the textual prompt. We apply this model to three small datasets: pixel art video game maps, video game sprite images, and down-scaled emoji images and apply novel augmentation strategies to improve the performance of our model on these limited datasets. We evaluate our models' performance using cosine similarity score between text-image pairs generated by the CLIP VIT-B/32 model to demonstrate quality generation.

Downloads

Published

2023-10-06

How to Cite

Merino, T., Negri, R., Rajesh, D., Charity, M., & Togelius, J. (2023). The Five-Dollar Model: Generating Game Maps and Sprites from Sentence Embeddings. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 19(1), 107-115. https://doi.org/10.1609/aiide.v19i1.27506