r/LocalLLaMA · · 1 min read

how to install llamacpp the better way to wrapping it in python ui (CPU use only) ?

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

i want the best installation that fit my use and my low-compute H.W , i want to run small to above small llm like "qwen" 2b ,4b and 27b , and "gemma" 31B. rely completely on only old CPU 4th.gen i7 with that few 32gb 'slow' ddr3. i will use llamacpp as python program with simple ui calling it like this from llama_cpp import lama ..so on.

should i install llamacpp like this : inside venv, pip install git+ggmlorg/llamacpp repo or other that made for CPU as ik_llamacpp ?

or : build like this without venv , git clone llamacpp repo; cd llama.cpp; cmake -B build; cmake --build build -j ?

or : install from pip inside venv : CMAKE_ARGS="-DGGML_CUDA=OFF" pip install llama-cpp-python ?

and is pip llamacpp differ from github repo nad why ?

, what is best for my use case ?

submitted by /u/BeautyxArt
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA