A handy llama-server launcher with easy model and configuration customisation
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
I wanted something that I could easily configure to manage a set of sensible defaults, that supports multiple llama-server binaries, with per-model over-rides, and command line over-rides.
The utility is here: https://github.com/stew675/start-llama
I know that llama-server has its own model loading configuration available via the API end-point, but I just wanted something that I could start from the command line easily in one step.
I don't know if anyone else may find this useful or not, but I'll share it here anyway in case someone does.
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.