TRANSFORMER EXPLAINER: Interactive Learning of Text-Generative Models

Authors

  • Aeree Cho Georgia Institute of Technology
  • Grace C. Kim Georgia Institute of Technology
  • Alexander Karpekov Georgia Institute of Technology
  • Alec Helbling Georgia Institute of Technology
  • Zijie J. Wang Georgia Institute of Technology OpenAI
  • Seongmin Lee Georgia Institute of Technology
  • Benjamin Hoover Georgia Institute of Technology IBM Research
  • Duen Horng (Polo) Chau Georgia Institute of Technology

DOI:

https://doi.org/10.1609/aaai.v39i28.35347

Abstract

Transformers have revolutionized machine learning, yet their inner workings remain opaque to many. We present TRANSFORMER EXPLAINER, an interactive visualization tool designed for non-experts to learn about Transformers through the GPT-2 model. Our tool helps users understand complex Transformer concepts by integrating a model overview and smooth transitions across abstraction levels of math operations and model structures. It runs a live GPT-2 model locally in the user’s browser, empowering users to experiment with their own input and observe in real-time how the internal components and parameters of the Transformer work together to predict the next tokens. 125,000 users have used our open-source tool at https://poloclub.github.io/ transformer-explainer/.
AAAI-25 / IAAI-25 / EAAI-25 Proceedings Cover

Downloads

Published

2025-04-11

How to Cite

Cho, A., Kim, G. C., Karpekov, A., Helbling, A., Wang, Z. J., Lee, S., Hoover, B., & Chau, D. H. (Polo). (2025). TRANSFORMER EXPLAINER: Interactive Learning of Text-Generative Models. Proceedings of the AAAI Conference on Artificial Intelligence, 39(28), 29625-29627. https://doi.org/10.1609/aaai.v39i28.35347

Issue

Vol. 39 No. 28: IAAI-25, EAAI-25, AAAI-25 Student Abstracts, Undergraduate Consortium and Demonstrations

Section

AAAI Demonstration Track