llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 14d ago
b10194
ggml-cuda: Allow transpose-free gemmv computation ( #26171 ) When matrix's weights are shaped 1xK is leverage a transpose-free computation to use mat_mul_vec_f. Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
20 -
b10192
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
27 -
b10189
Remove custom cpu op from the M3 graph, express with stock ops ( #26297 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu…
23 -
b10188
metal: fix memory unwire if model is freed without any GPU operations ( #26082 ) metal: fix memory leak if model is freed without any GPU operations metal: run dummy work only if residency sets are used metal: wrap function in #if defined metal: measure system-wide wired memory…
6 -
b10186
ggml : Fix issue with kleidiai ci and stringop overflow warning ( #26277 ) Signed-off-by: Jonathan Clohessy [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
28 -
b10184
mimo2: address MTP review feedback ( #26228 ) Co-authored-by: tnhnyc [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
38 -
b10182
llama: move suppress_tokens handling to common/sampling ( #26276 ) llama: move suppress_tokens handling to common/sampling address security issues rm has_logit_bias Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
23 -
b10181
ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory ( #26141 ) ggml_cuda_should_use_mmq() selects MMQ purely from the quantization type. The current MMQ configurations are designed and maintained against a minimum of 48 KiB per-block shared memory, the limit…
36 -
b10180
sycl: contiguous fast path + 32-bit index math for unary elementwise ops ( #25946 ) sycl: contiguous fast path + 32-bit index math for unary elementwise ops sycl: use fastdiv for elementwise index math Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
8 -
b10179
vendor: update BoringSSL to 0.20260728.0 ( #26241 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
26 -
b10178
server : add trace logging for slot similarity checking ( #26271 ) Adds trace logging in server-context.cpp for slot similarity checking during prompt cache slot selection, including skip reasons and similarity calculation details. Assisted-by: llama.cpp:Qwen3.6-27B Website:…
21 -
b10176
RPC: add tensor_memset ( #25912 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
37 -
b10175
add rdna3.5, and 3 to mmq configs so they can be tuned independently. ( #26199 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
26 -
b10174
model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) ( #25980 ) model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) Adds GLM-5.2 NextN/MTP as a --spec-type draft-mtp target: nextn tensor loading via the qwen35moe/step35-style presence…
29 -
b10173
model: Add Laguna-S-2.1 LLM_TYPE ( #26233 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
5 -
b10172
ggml-webgpu: Fix some binding alias issues to support all archs, fix recurrent-state-rollback test ( #25931 ) Add overlap glu variant to support all archs, fix recurrent-state-rollback test format Fix all arch overlapped ranges format diagnose bus error on apple ci More testing…
25 -
b10171
opencl: skip the Adreno KQ/KQV image kernels for multi-stream batches ( #26189 ) The Adreno KQ/KQV image1d kernels (ggml_cl_mul_mat_kq_kqv_adreno) ignore dim 3 entirely: the sub-buffer covers only nb02*ne02 bytes and the kernel receives no ne03/ne13/nb03/nb13 arguments. With the…
19 -
b10167
server: abstract llama_memory calls to common_memory ( #26221 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
26 -
b10166
ggml : set output of view src ( #25729 ) llama-graph: set_outputs to t->view_src change set_output to GGML_ASSERT about views not being outputs sampler : avoid views in outputs cont : fix dist sampler cont : consistent logits handling ggml : set output of view src graph :…
28 -
b10165
vulkan: add iq4_nl support back to FA ( #24585 ) vulkan: add iq4_nl support back to FA I was originally concerned about wasting shared memory on the LUT, but it's small and unlikely to matter in practice. Also support q1_0 for non-coopmat2. Fixes #23681 remove q1_0 FA support…
36 -
b10164
ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration ( #22675 ) ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration cuda: added SSD CICD fixes for CUDA / HIP / MUSA / MSVC. ggml-cuda: review comments fixed. ggml-cuda: Fuse M matrix materialization…
23 -
b10158
spec: add eagle3-v3 support for gpt-oss model ( #25794 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
23 -
b10156
Disable -ffast-math on HIP ( #25495 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan)…
5 -
b10154
common : add common_print_available_devices() ( #26170 ) Signed-off-by: Adrien Gallouët [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
13 -
b10153
model: Add support for Nanbeige4.2 ( #25994 ) support nanbeige4.2 model fix fix flake8 Lint check fix loop bound check and drop redundant head_dim Co-authored-by: root [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
18 -
b10152
fit : count nextn (MTP) blocks in n_gpu_layers so front layers stay on GPU ( #26177 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
8 -
b10151
sycl(build): parallelize ocloc invocations ( #25903 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
5 -
b10150
ggml : adjust logic for offloading ops to weight's backend ( #25832 ) ggml : adjust logic for offloading ops to weight's backend llama : dsv4 graph fixes Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
36 -
b10149
tests : remove unnecessary sync in test-save-load-state ( #26166 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
34 -
b10148
common: fix explicit -md precedence over draft sidecar resolution ( #26165 ) common: fix explicit -md precedence over draft sidecar resolution Follow-up of #25955 , an explicit --model-draft file given with -hfd was silently overridden by the sidecar resolution of the draft…
28 -
b10146
ggml-cpu: Enable BF16 tiled gemm optimization on PowerPC ( #26068 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
9 -
b10145
args: add -lm mlock where it mlocks but doesnt mmap ( #26135 ) arg: add -lm mlock where it mlocks but doesnt mmap Signed-off-by: Aaron Teo [email protected] docs: rm unwanted docs changes Signed-off-by: Aaron Teo [email protected] docs: revert auto-formatting Signed-off-by:…
35 -
b10144
server + ui: fix stream routes for model names containing a slash ( #26137 ) server + ui: refactor resumable stream routes to query string conv_id The conversation id can embed a model name containing slashes (ggml-org/...) in router mode, which the decoded path splits before…
37 -
b10142
mtmd: Add Vision Support for Minimax-M3 ( #25113 ) Add preliminary MiniMax-M3 support Text-only port that re-uses existing components: MiniMax-M2 style GQA with per-head QK-norm and partial rotary, DeepSeek-V3 style leading-dense and routed/shared experts, and swigluoai…
18 -
b10141
mtmd: fix android build ( #26150 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
21 -
b10133
ui: fix context gauge card regressions and land at the conversation e…
5 -
b10121: ui: reduce per-token render cost when streaming (#26053)
performance harness - the empirical root Assisted-by: Claude Opus 4.8 210.36ms -> 2.67ms per streamed token Assisted-by: Claude Opus 4.8 11.58ms -> 0.62ms per streamed token Assisted-by: Claude Opus 4.8 22.02ms -> 3.33ms per streamed token Assisted-by: Claude Opus 4.8 3.07ms ->…
6 -
b10107
hexagon: fix Windows crash when op_poll is enabled ( #26029 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
8 -
b10106
CUDA: fix external compilation of q1_0 MMQ ( #25778 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
4 -
b10105
args: refactor mlock/mmap/directio into load-mode ( #20834 ) args: overhaul mmap/mlock/dio into single arg Signed-off-by: Aaron Teo [email protected] docs: update docs with llama-gen-docs Signed-off-by: Aaron Teo [email protected] chore: satisfy code quality Signed-off-by:…
22 -
b10108
hexagon: further improved pipeline of the core bits (L2, DMA, MM, FA)…
38 -
b10099
CUDA: Improve NVFP4 W4A4 activation quantization ( #25730 ) Squash history before conflict-resolution during rebase on master WIP commit Add 32-byte loads, restore per-block amax Use nvfp4x4 intrinsic when available Fuse per-channel amax and quantization kernels Do pointer…
6 -
b10098
hexagon: activation ops update ( #25974 ) hex-geglu: optimized all-in-one geglu microkernel hex-geglu: enable non-contiguous src and strided DMA hex-act: enable non-contiguous srs and strided DMA for rest of ACT ops hex-act: generalize GLU per-thread functions via…
28 -
b10094
common: infer the speculative type from the draft repo sidecars ( #25989 ) With -hfd pointing to a repo that ships mtp-/dflash-/eagle3- sidecars and no --spec-type given, the draft resolved to a full model while the sidecar was the intended draft. When the speculative types are…
19 -
b10093
Fix DeepSeek4 crafted template ( #25414 ) chat: fix DS4 template to explicitly follow reference behavior Support DeepSeekv4 flag ( drop_reasoning ). fix: hook DS3.2 parser for DS4 as well fix: add tool result reordering fix: post-merge Website: https://llama.app macOS/iOS: macOS…
20 -
b10092
ggml: enable PowerPC backend variants on AIX ( #25983 ) ggml: enable PowerPC backend variants on AIX Allow the PowerPC CPU backend variants to be built on AIX by extending the platform check in the CMake configuration. This reuses the existing PowerPC backend implementations…
5 -
b10103
metal : add f16 type support to leaky relu ( #25981 )
23 -
b10091
ci : fix SYCL package shared library lookup ( #25987 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
5