r/LocalLLaMA · · 1 min read

A handy llama-server launcher with easy model and configuration customisation

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

I wanted something that I could easily configure to manage a set of sensible defaults, that supports multiple llama-server binaries, with per-model over-rides, and command line over-rides.

The utility is here: https://github.com/stew675/start-llama

I know that llama-server has its own model loading configuration available via the API end-point, but I just wanted something that I could start from the command line easily in one step.

I don't know if anyone else may find this useful or not, but I'll share it here anyway in case someone does.

submitted by /u/Look_0ver_There
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA