Introducing Quartermaster, an open source local AI platform designed for ease of use that does not sacrifice customizability

It started as a fork of llama-swap, but I have been building it out for myself since then as a convenient tool for all my local AI needs, and by now it has drifted far enough to be its own thing.

The main idea is that you point it at your models folder and it configures things for you. It reads the GGUF headers, measures how much VRAM you actually have free, and works out context length, GPU offload, CPU/MoE split and KV cache size per model. All of it stays editable per model if you disagree with what it picked.

It is not only text. llama.cpp for LLMs, with the Vulkan, CUDA, ROCm or CPU build downloaded and kept updated for you, stable-diffusion.cpp for images (SD, SDXL, Flux, Qwen-Image, LoRAs, upscaling), and vLLM if you already have it set up. You can register any other backend yourself by pointing at an executable, which is how I run TTS, and how you would run a llama.cpp fork like ik_llama. Everything sits behind one OpenAI-compatible API on one port, with a single scheduler, so models swap in and out without fighting each other for VRAM.

There is also a chat playground built in with web search, and a Hugging Face browser to search for a model, pick a quant and download it straight into the models folder and much more!

If you are interested, you can read more about it here. MIT licensed.