llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 6d ago
b10318
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
13 -
b10313
server: (router) add LRU scheduler ( #26572 ) add lru_sched handle coalescing (req leaves waiting queue) add tests fix stream case address review comments Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
35 -
b10312
server: (router) do not evict busy models ( #26567 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
37 -
b10311
mtmd: stop feeding the text stream again during Qwen3-TTS generation ( #26706 ) The reference implementation has two mutually exclusive prompt layouts. In non streaming mode the prefill carries the whole utterance text plus tts_eos summed with codec_pad, and the trailing text…
29 -
b10310
ggml : add aarch64 HWCAP fallbacks and fix fp16 variant detection ( #25554 ) ggml : add fallback definitions for missing aarch64 HWCAP bits ggml : require HWCAP_ASIMDHP for the aarch64 fp16 cpu variants Also rename has_fp16_va to has_fp16, the field gates the whole FEAT_FP16…
18 -
b10308
Mitigate crashing issue on Windows MSYS2 UCRT64 environment (GCC 16.1.0) ( #26555 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
16 -
b10307
sycl: fix UE4M3 parsing ( #25608 ) The NVFP4 quantization format stores a scaling factor for every group of 16 weights, packed into a single UE4M3 byte. The SYCL GPU code was converting these scale values using the E4M3 path, but that's signed , and these are unsigned values.…
13 -
b10306
sycl: *glu flat path ( #26354 ) tests: add SWIGLU perf cases perf mode had no GLU coverage. Adds SWIGLU at 17408 columns, 512 and 2048 tokens, f16 and f32, with the operands both fused and split. sycl: consolidate fused-GLU kernels They differed only in which op_* they called,…
14 -
b10305
sycl : Support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PRE ( #26568 ) support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PREwq update ops.md fix format issue Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
8 -
b10303
sycl : fix error Error OP FLASH_ATTN_EXT on arc770 ( #26441 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
17 -
b10301
cuda: fix warnings for unused variable/function ( #26688 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
36 -
b10299
metal : avoid threadgroup matrix array instantiation in kernel_lightning_indexer ( #26646 ) In MSL, declaring an array of matrix types like threadgroup half4x4 causes a 'no matching constructor' compilation error because MSL matrix types do not have zero-argument default…
17 -
b10298
mtmd: add chunk save/load function ( #26645 ) mtmd: add chunk save/load function nits add tests rn _MAX --> _COUNT Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
11 -
b10297
server: fix empty response for /cors-proxy ( #26656 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
10 -
b10295
model-loader : fix quantized reshaped tensor strides ( #26672 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
32 -
b10293
ci : onboard AMD ROCm CI with gfx1151 fixes ( #26544 ) ci: prepare for amd rocm ci Signed-off-by: Aaron Teo [email protected] ci: fix editorconfig-checker Signed-off-by: Aaron Teo [email protected] ci: fix device not recognised Signed-off-by: Aaron Teo [email protected] ci:…
23 -
b10291
vulkan: fix submission batching size, add debug tools for diagnosing causes of DeviceLost drivers errors ( #26371 ) vulkan: add debug tooling to get more information about a DeviceLost error fix submission threshold applied too late use logging macros, throw instead of aborting…
18 -
b10290
mtmd/ggml: add ggml_build_forward_order ( #26649 ) ggml: add ggml_build_forward_order ggml_build_forward_expand marks the tensor and all its ancestors for compute, so using it as a pure ordering hint (keeping q, k and v together) defeats ggml_build_forward_select: the unselected…
32 -
b10289
server: harden the file_glob_search directory walk ( #26626 ) server: don't walk Windows junctions in file_glob_search std::filesystem reports a junction as a plain directory, so the symlink guard misses it and a junction pointing back at an ancestor is walked until the path…
13 -
b10288
tests: re-enable MiniMax M3 in test-llama-archs ( #26633 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
14 -
b10287
mtmd: Unlimited-OCR fix max_tiles, setting in converter ( #25614 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
23 -
b10286
grammar : degrade max repetition >= 2000 to unbounded ( #26613 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
5 -
b10285
mtmd: support multi-row batching for deepseek-ocr ( #26154 ) mtmd: support multi-row batching for deepseek-ocr mtmd: weave deepseek-ocr rows in one shot instead of per row ( #26615 ) Co-authored-by: Saba Fallah [email protected] Website: https://llama.app macOS/iOS: macOS…
15 -
b10284
fit: Fix memory allocation for MTP layers ( #26605 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
29 -
b10282
server: Adding spec-decode counters to /metrics endpoint ( #26389 ) server: add spec-decode counters to /metrics endpoint server: fixed review comments and now aligned param names exactly with vLLM. Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
28 -
b10280
vendor : apply patches for subprocess.h ( #26606 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
29 -
b10278
build : remove GGML_METAL_USE_BF16 from all build scripts ( #26604 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
30 -
b10276
Prefer npm ci over install for security ( #26601 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
13 -
b10275
server: decode Windows OEM output to UTF-8 in built-in tools ( #26597 ) a child process writes in the OEM code page, which is not UTF-8 on a western Windows install, so accented output reaches the JSON layer as invalid bytes and gets replaced there, silently losing the…
38 -
b10274
mtmd: correcting duplicate empty audio chunks for short inputs ( #26536 ) correcting duplicate empty audio chunks for short inputs tests.sh code restored Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
30 -
b10273
sampler : remove "full-context windows" from history-based samplers ( #26524 ) Resolve -1 to 1024 instead of ctx-len for samplers Because of backend-sampling we initialize samplers before the complete llama_context is there. Therefore, we cannot infer the resolved context length…
36 -
b10271
ui: CWD for agent ( #26518 ) server : extend file_glob_search for UI pickers ui : add per-conversation working directory with picker ui : add path navigation and search scope to cwd picker Treat path-like queries (starting with / or ~) as directory navigation instead of…
15 -
b10270
mtmd: support Qwen3-TTS (note: breaking change to llama-tts binary) ( #26254 ) convert text model main model load ok convert encoder ok speaker encoder loading ok speaker enc graph adapt vocab for backbone (with some tricks) add suppress_tokens poc new mtmd gen api convert…
17 -
b10269
models : fix dflash wo_a reshape on load ( #26577 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
34 -
b10268
ci: fix pre-built binaries no longer working on macOS 15 and below ( #26375 ) ci: fix pre-built binaries no longer working on macOS 15 and below ci: add macOS deployment target to disabled KleidiAI build Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS…
5 -
b10267
speculative : refactor enabled configs common_speculative_init ( #26510 ) This commit contains a suggestion to reduce some code duplication in common_speculative_init when adding the enabled speculative decoding configurations. No tests were added but the existing server tests…
8 -
b10265
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
22 -
b10262
vulkan backend ops: implemented GATED_LINEAR_ATTN ( #25601 ) vulkan : add GATED_LINEAR_ATTN op docs : update Vulkan ops vulkan : remove unused GLA spec constant Updated ops.md ops.md update Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
29 -
b10261
vocab : validate plamo2 byte tokens ( #26511 ) validate plamo2 byte tokens --typo Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
32 -
b10259
model : allow reshape of tensors during load ( #26531 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
9 -
b10258
llama : move n_vocab from llama_sampler_data to penalty_sampler ( #26520 ) This matches how it is done for logit_bias and mirostat samplers, see #25262 (comment) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
13 -
b10256
sycl: parallelize the non-contiguous concat kernel ( #25852 ) sycl: parallelize the non-contiguous concat kernel Launch geometry only: the non-contiguous concat kernel launched a single-lane work-group (1, 1, 1), now it will launch a (1, 1, SYCL_CONCAT_BLOCK_SIZE) one.…
30 -
b10255
Extended SYCL oneDNN SDPA to non-FP16 KV caches (Q4_0–Q8_0 and FP32) ( #25874 ) sycl: extend oneDNN SDPA to Q4_0-Q8_0 and F32 KV caches Extends the oneDNN SDPA path (PR #25222 ) to handle non-F16 KV caches by dequantizing or converting K/V to dense FP16 on-device before feeding…
4 -
b10254
chat : add new template for DeepSeek V4 Flash 0731 ( #26398 ) common/chat: update DeepSeek V4 templates Align the DeepSeek V4 templates with the official encoders while keeping parser behavior out of this change. Default drop_thinking for DeepSeek V4 history so prior thinking is…
37 -
b10253
vendor : update cpp-httplib to 0.52.0 ( #26485 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
23 -
b10252
vendor : update BoringSSL to 0.20260803.0 ( #26523 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
14 -
b10251
model : support MTP in GLM-4.7-Flash ( #24868 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
37 -
b10250
tests: add model resolution test on synthetic repo listings ( #26172 ) tests: add model resolution test on synthetic repo listings Include download.cpp and arg.cpp inside a namespace with hf_cache monkey patched to serve hardcoded listings, so the resolution and the model…
36 -
b10249
server: add get_info tool ( #26522 ) server: add get_info tool fix --rpc in docs server: harden get_info probe result handling Report the OS as unknown when the probe process fails to spawn or times out, so the diagnostic text from run() is never returned as an OS name. Strip…
10