I got a real transformer language model running locally on a stock Game Boy Color!

No phone, PC, Wi-Fi, link cable, or cloud inference.

• The cartridge boots a ROM, and the GBC runs the model itself.
• The model is Andrej Karpathy’s TinyStories-260K, converted to INT8 weights with fixed-point math so it can run without floating point.
• Built with GBDK-2020 as an MBC5 Game Boy ROM.
• The model weights live in bank-switched cartridge ROM. Prompt entry happens on-device with the D-pad/buttons and an on-screen keyboard.
• The prompt is tokenized on the Game Boy, then the ROM runs transformer prefill + autoregressive generation. The KV cache is stored in cartridge SRAM, because the GBC’s work RAM is tiny.

It is extremely slow, and the output is gibberish because the math is heavily quantized/approximated, but the core thing works!

Hardware: stock Game Boy Color + EZ Flash Junior + microSD.

Used Codex for a large portion of the building!

https://github.com/maddiedreese/gbc-transformer