Built Kivarro, an all-in-one local inference workbench. Looking for brutal feedback from people who actually run models locally.

I’ve been building Kivarro, a desktop app for local model inference.

The idea is simple:

Most local LLM tools solve one slice of the workflow.

One app for chat. One app for model files. One script for llama.cpp flags. One dashboard for memory. One terminal for logs. One random note somewhere for benchmark results.

I wanted all of that in one place.

So Kivarro is my attempt at an all-in-one local inference app. Not a hosted service. Not a wrapper around a cloud API. A local-first desktop workbench for people running models on their own machines.

Demo attached.

What it does right now:

• local model registry for GGUF / safetensors / bin / mlx files
• GGUF metadata reading for architecture, quantization, context length, layers, etc.
• llama.cpp / llama-server supervision
• mistral.rs backend option
• OpenAI-compatible local API view
• streaming chat completions
• profile switching for different workflows
• expert tuning for sampling/runtime settings
• context length, KV cache, batch, micro-batch, CPU threads, GPU layers
• hardware fit / memory planning before loading
• RAM + context + token/sec visibility while running
• benchmark view for tokens/sec and load timing
• local RAG knowledge-base workbench
• persisted logs/settings/profiles
• cross-platform builds: Windows, Windows ARM64, macOS Intel, macOS Apple Silicon, Linux x64, Linux ARM64

The bigger vision is:

A serious local inference cockpit.

Not just “type prompt, get answer.”

I want Kivarro to become the place where you can:

• import a model
• understand if it fits your hardware
• tune it without memorizing every backend flag
• run it locally
• expose it through an API
• benchmark it
• test retrieval
• save reusable profiles
• see exactly what is happening while it runs

I’m not claiming it is perfect. It is early. Builds are unsigned. The RAG part is currently a workbench, not automatic prompt injection. Agents are still a draft/control-plane area. The app is source-available under a non-commercial license.

What I want from this sub is feedback from people who actually run local models:

• What is missing from the workflow?
• What would make this useful enough for your daily local inference setup?
• Are the tuning controls the right ones?
• Is the memory/context display showing the right information?
• Should I prioritize Ollama/vLLM/MLX support next?
• What would make you trust this over your current terminal/scripts setup?

I know r/LocalLLaMA is allergic to empty promo posts, so I’ll be direct:

I built this. I want it to be useful. I’m looking for criticism before I build the next layer.

Repo link: https://github.com/AKMessi/kivarro