llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 10d ago
b10249
server: add get_info tool ( #26522 ) server: add get_info tool fix --rpc in docs server: harden get_info probe result handling Report the OS as unknown when the probe process fails to spawn or times out, so the diagnostic text from run() is never returned as an OS name. Strip…
10 -
b10248
vocab : validate default special token ids ( #26506 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
14 -
b10247
ggml: use dynamic allocation for split graph inputs ( #22789 ) ggml: use dynamic allocation for split graph inputs Replace fixed-size GGML_SCHED_MAX_SPLIT_INPUTS arrays with dynamically allocated buffers in the backend scheduler. This fixes crashes when loading wide MoE models…
7 -
b10246
opencl: route large q6_K lm_head to the flat GEMV ( #26427 ) add a direct size condition for large weights; the original dimension condition is insufficient -- q6_K lm_head for gemma-4 E2B has [1536, 262144], which is big enough to slowdown gemv_noshuffle but does not satisfy…
25 -
b10245
graph : fix unused input tensors in minimax m3 graph ( #26519 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
23 -
b10244
model: M3: Move MSA into a new memory implementation ( #26338 ) Move MSA logic from llama-kv-cache into llama-kv-cache-msa cont : minor cont : ws fix Co-authored-by: Georgi Gerganov [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
36 -
b10243
llama : allocate indexer cache only in "full" indexer layers ( #26474 ) Co-authored-by: Stanisław Szymczyk [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
4 -
b10242
CUDA: Add backend sampler for penalties sampler ( #25262 ) sampling: enhance penalty handling in common_sampler_init Set default value for penalty_last_n based on model context if not specified. Ensure penalty_last_n and n_prev are non-negative. Update llama_sampler_penalties…
20 -
b10241
CUDA: Fix data-races when reusing SMEM in block_reduce ( #26385 ) CUDA: Fix data-races when reusing block_reduce block_reduce currently doesn't resync after reading from SMEM, causing potential data-races when reusing SMEM for multiple reductions. One may consider simply always…
24 -
b10240
server: add notice for upcoming default port change 8080 --> 9931 ( #26508 ) server: add notice for upcoming default port change 8080 --> 6631 add link to PR correct to 9931 Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
21 -
b10238
model: MTP support for Qwen3-Next ( #25589 ) mtp for qwen3nex fix for python type-check Fix to compute num_mtp from directly mtp layer define opt_num_mtp_layers in _QwenMtpMixin and fix some comments Fix for python type check Update gguf-py/gguf/constants.py Co-authored-by:…
26 -
b10237
llama : MTP support for DeepSeek V3.2 ( #26457 ) llama : MTP support for DeepSeek V3.2 model : no need to include MTP layers during DeepSeek V3.2 model type discovery Co-authored-by: Stanisław Szymczyk [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon…
10 -
b10236
metal: implement DSv4 Lightning Indexer ( #25893 ) metal: implement F16 Lightning Indexer Implement GGML_OP_LIGHTNING_INDEXER for 128-dimensional, 64-head inputs with F32 queries and weights plus F16 keys and masks. Add tiled and tail kernels and test KV lengths around 8- and…
25 -
b10235
metal : add SILU_BACK ( #25982 ) feat(silu_back): implemented silu_back op for f32 fix(silu_back): removed redundant asserts in ggml-metal-ops.cpp function ggml_metal_op_silu_back. Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
38 -
b10234
metal : add F16 support for bin ops ( #26465 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
28 -
b10233
opencl: limit local workgroup size for GLU operation ( #26383 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
27 -
b10232
metal: implement DeepSeek V4 hyper-connections ( #26459 ) Implement GGML_OP_DSV4_HC_COMB, GGML_OP_DSV4_HC_PRE, and GGML_OP_DSV4_HC_POST with SIMDgroup register and shuffle optimized kernels. Add Metal dispatch and support plumbing and test the production Sinkhorn iteration count…
10 -
b10231
common: support the DSpark sidecar resolution ( #26458 ) The dspark- files resolve like the other speculative sidecars: the -hfd tag applies to them, a requested sidecar resolves without a full model at the tag, and an explicit -md selection disables the discovery. When no type…
33 -
b10229
opencl: bugfix increment ref_count in ggml_backend_opencl_init() ( #26162 ) Incrementing ref_count at the beginning is important later in the free() method of the ggml_backend_opencl_context at program end. If we do not increment the ref_count , the result would be -1 here, and…
15 -
b10228
DeepseekV4 MTP + DSpark ( #25784 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
35 -
b10227
chat : add qwen3 specialized parser ( #26252 ) Add tagged thinking tool parser chat : refactor and add permute helper cont : add support for <tool_call> omission cont : update tool delimiters cont : add comment for qwen3-coder cont : fix trigger pattern for <function…
25 -
b10225
model : load MiMo V2 MTP tensors only if used ( #26412 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
26 -
b10226
sycl: fix classification of iGPUs ( #26105 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
14 -
b10224
ggml-webgpu: add support for f16 repeat ( #26307 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
15 -
b10223
test: fix some CI errors ( #26415 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
8 -
b10221
vendor : update BoringSSL to 0.20260730.0 ( #26353 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
13 -
b10219
cli : persist reasoning_content in chat history ( #26362 ) cli : persist reasoning_content in chat history llama-cli collected reasoning from the stream for display but only stored assistant content in messages, so --reasoning-preserve could not re-inject prior thoughts on later…
22 -
b10218
mtmd: add minicpmv46 downsample ( #25993 ) add minicpmv46 downsample Signed-off-by: tc-mb [email protected] put downsample mode inside gguf. Signed-off-by: tc-mb [email protected] build mtmd_image_preprocessor_llava_uhd Signed-off-by: tc-mb [email protected] fix…
12 -
b10217
chat : enable tool call in thinking for DS4 ( #26269 )
33 -
b10216
vulkan: add POOL_1D op ( #25431 ) vulkan : add pool1d push constants and pipeline field Declared data structures needed for POOL1D OP, which are the vk_op_pool1d_push_constants struct and pipeline_pool1d_f32 field. vulkan : add pool1d compute shader Added pool1d.comp for Vulkan…
23 -
b10214
mtmd: add n_embd_head ( #26342 ) Co-authored-by: Daniel Han [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
10 -
b10213
Support rotated kv cache quant ( #26180 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan)…
15 -
b10212
llama : load MTP tensors only if they are really used ( #26296 ) llama : load MTP tensors only if they are really used llama : skip loading MTP (if not used) in remaining models that support MTP Co-authored-by: Stanisław Szymczyk [email protected] Website: https://llama.app…
35 -
b10211
vulkan: update vulkan sdk to 1.4.357.0 ( #26303 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
27 -
b10210
server: correct accepted tokens when need draft token replay ( #26320 ) spec: correct accepted tokens when need draft token replay cont : naming Co-authored-by: Georgi Gerganov [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
34 -
b10209
cuda: extract Q2_0 elements via __byte_perm ( #25603 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
8 -
b10208
SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt proc… ( #25025 ) SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt processing fattn-mkl: fix interleaved dst layout in normalize kernel Fix mkl_fa_normalize_head: use interleaved dst layout ((query *…
36 -
b10207
[SYCL] support the missed types in cpy ( #26005 ) support the missed types in cpy use correct funct rm unused code Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
29 -
b10206
llama : enforce the same K and V cache types for DeepSeek V4; enable FA if V cache is quantized ( #25871 ) llama : enforce the same K and V cache types for DeepSeek V4; enable FA if V cache is quantized llama : enforce the same K and V cache types for MLA models Co-authored-by:…
23 -
b10205
ggml-zendnn : group matmul direct API for mul_mat_id ( #25918 ) ggml-zendnn : group matmul API for mul_mat_id ggml-zendnn : scale MUL_MAT_ID fallback threshold by expert count Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
9 -
b10204
sycl : support dev2dev memcpy by DEV2DEV_MEMCPY_FORWARD ( #26234 ) Co-authored-by: Neo Zhang Jianyu [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
9 -
b10203
[SYCL] Support q2 mul_mat ( #26231 ) support q2_0 in mul_mat support more q2_0 case Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
10 -
b10202
sycl: fuse RMS_NORM + MUL ( #26015 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan)…
29 -
b10201
ggml-webgpu: improve flash_attn_vec for quantized KV at long contexts ( #25956 ) improve fa of quantized kv cache Fix some bugs and some comments. fix v type check and some comments Fix build error caused by rebasing editorconfig checking pass Website: https://llama.app…
13 -
b10199
server: support inp embd to generate next token ( #26313 ) server: support embd for sampled token fix ~server_batch() Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
10 -
b10200
mtmd: add lanczos resize method [no release] ( #26341 )
21 -
b10198
vulkan: Support quantized concat ( #25684 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
21 -
b10197
Test support for alternative conv layout ( #25617 ) add bool cwhn = true to conv_2d test cases add layout check at graph building time extend layout checks for conv2d.cu kernel in CPU back-end kernel needs to be stored contiguously to prevent test failures with cwhn=1 trim white…
6 -
b10196
llama-context : sync pending async copies before clearing embd_seq ( #25676 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
20 -
b10195
tests : avoid building get-model.cpp many times ( #26317 ) tests : remove get-model.cpp tests : fix quant type selection Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
13