I built a serverless hosting platform for LoRA adapters with vLLM

It’s always bothered me that after fine-tuning a model for a project, there isn’t a particularly easy way to host it without either running it locally and keeping a GPU on 24/7 or paying for an entire GPU server.

There are managed options for LoRA serving on top of vLLM (AWS), but you generally still end up paying for an entire instance.

I started wondering: if 99%+ of the model weights are identical between the base model and something like a rank 8–32 LoRA/QLoRA adapter, why does each adapter need its own server?

vLLM already has a lot of the pieces needed to make this work efficiently, including batching, memory management, and fused LoRA kernels.

So I built Lorivo around the idea of having one GPU server per base model that can be shared between many LoRA adapters.

The basic flow is:

1. Upload a LoRA adapter
2. Lorivo identifies the base model and sends it to a GPU server running that model
3. The adapter is loaded into memory
4. You get an OpenAI-compatible endpoint for inference / usage directly from the web app

I also built a CLI to make uploading/deploying an adapter as simple as possible, along with a web interface for manual uploads. It's as easy as lorivo login api-key and lorivo deploy ./my-lora

The project is live: https://lorivo.dev

Right now I’m hosting the traffic myself on my own GPU, so the available inference model is limited to Qwen 3.5 4B which will always be 100% free

I also have around $1,000 in AWS credits that I’d like to use to add some additional models. I'm not sure which ones would be most useful to the community, though.

What models would you want to see hosted?

For Qwen 3.5 4B, you can also use the base model without an adapter. Just set the model to qwen3-4b instead of qwen3-4b:my-adapter. I'm currently allowing a 32k context window.

I’m also intentionally not saving chats or inference requests. The only usage data collected is token count and timestamp.

Would love to get feedback from people here, especially on the architecture, which models would be useful to host, and whether this solves an actual problem you've run into.