I got Gemma 4 running directly inside Godot using only GDScript and Vulkan compute shaders

I wanted to see if an LLM could run inside Godot without llama.cpp, Python, a server, or a GDExtension.

It works. This Godot 4.7 project runs gemma-4-E2B-it-Q4_K_M.gguf locally. The model calculations run in Vulkan compute shaders, while GDScript handles GGUF loading, tokenization, sampling, the KV cache, and the chat UI.

It is only an experiment. It supports this one model and is about 10× slower than llama.cpp with CUDA. Still, I found it interesting that this was possible using only Godot.

Code: https://github.com/asallay/godot-llm