llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 1h ago
b10426
ggml: force single thread on wasi ( #25686 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
27 -
b10425
sycl: fuse the gated-delta-net state writeback cpy ( #26643 ) Port of #23940 . Arc Pro B70, Qwen 3.6 27B Q4_K - Medium (48 of its 64 blocks run gated_delta_net), -ngl 99 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 2048, interleaved A/B passes of r=3: tg128 23.91 / 23.90 / 23.90 -> 24.19…
26 -
b10424
dflash : clarify output logging of target_layer_ids ( #27013 ) This commit tries to make the logging of target_layer_ids a bit clearer and easier to read. Currently the output generated looks like this: 0.00.468.624 D load_arch_hparams: DFlash extract_layers = [0.00.468.626 D 2,…
25 -
b10423
common: apply CPU parameters across tools ( #27026 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
9 -
b10419
OpenVINO: Qwen3.5, memory optimization, and test-recurrent-state-rollback ( #26952 ) OpenVINO backend: 1) enable gpt-oss moe on OV bk; 2) enable mxfp4 support OpenVINO backend: disable TOPK_MOE op test OpenVINO Backend: Add op FILL support OpenVINO backend: enable set rows with…
36 -
b10418
[SYCL] Support host pinned mem to improve SYCL Host-to-Device Memory Access ( #26789 ) support host pinned mem, ggml_backend_sycl_host_buffer_type_get_max_size, fix the thread-safe issue Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
6 -
b10417
chat : fix LFM2 tool call arg name prefix ambiguity ( #26960 ) Assisted-by: Claude Opus 5 Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu…
6 -
b10416
server : serve index.html with no-cache ( #27006 ) index.html was served with max-age=31536000, immutable like the hashed assets, but its name is stable while its contents change every build, so a cached copy pins the UI to an old build. It now revalidates via its existing ETag,…
7 -
b10415
spec : auto-detect mtp draft model type ( #27005 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
37 -
b10414
metal : add TQ2_0 support ( #26980 ) metal: add TQ2_0 support Add support for the GGML_TYPE_TQ2_0 (ternary, 2 bits per element) type in the Metal backend. Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731 cont : optimize mul_mv kernel float ops over integer ops precalculate sums…
29 -
b10413
common : auto-detect spec type from draft GGUF metadata ( #26814 ) common : auto-detect spec type from draft GGUF metadata When -md loads a local draft model without --spec-type, the sidecar inference in common_models_handler_apply only checks HF repo sidecars and misses local…
31 -
b10412
spec: enable backend sampling for both dflash & dspark ( #26958 ) dflash: enable backend sampling for both dflash & dspark enable p_min > 0 in backend sampling and add guard cont : add TODO Co-authored-by: Georgi Gerganov [email protected] Website: https://llama.app macOS/iOS:…
21 -
b10411
ggml-cpu/ops: vectorize flash-attention V-cache F16 to F32 conversion ( #26947 ) Co-authored-by: jinzihao [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
9 -
b10410
sycl: remove separate fp32 type promotion in gemm non-oneDNN path ( #26372 ) sycl: use automatic fp16 promotion in gemm sycl: remove redundant comment Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
22 -
b10408
sycl : Add DMMV ESIMD Q3_K kernel ( #26251 ) Add DMMV Q4_K and Q6_K ESIMD kernels Configure cmake build with -DGGML_SYCL_ESIMD=ON to enable. Signed-off-by: Todd Malsbary [email protected] Refactor ESIMD kernels to share common code Signed-off-by: Todd Malsbary…
16 -
b10405
ggml-hip : remove -funsafe-math-optimizations ( #26696 ) It enables -fassociative-math, which reassociates FP reductions and can flip greedy argmax on RDNA3.5 (e.g. MTP speculative decode diverging from the non-speculative baseline). Drop it so HIP builds are IEEE-conformant.…
11 -
b10400
ggml : fix arm builds, unused var ( #26991 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
18 -
b10398: common: add system-level config file (#26118)
common: Add CLI > ENV > models-presets > INI precedence CLI flags have the highest precedence ENV vars have the second-highest precedence System and User configs have the lowest precedence Linux/BSD/Mac /etc/llama.cpp/config.ini <…
21 -
b10375
chat : tighten bare function parsing for Qwen models ( #26793 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
28 -
b10373
imatrix.cpp: Move finite check and only check touched experts ( #26861 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu…
5 -
b10369
mtmd: support pocket-tts ( #26871 ) adapt the api text model ok working impl, need verify and clean up mtmd: build the pocket-tts transposed convolutions as GEMM + col2im ggml_conv_transpose_1d has no grouped mode, so the depthwise upsample was built as one convolution and one…
16 -
b10380: chat : fix muse-glimmer detection of tool calls after EOM (#26879)
chat : fix muse-glimmer swallowing a trailing tool call into content Muse Glimmer routinely answers the user and calls a tool in a single generation. The template terminates a message with <|eom|> when more messages follow in the same turn and <|eot|> only at the end of the…
29 -
b10361
model : fix SWA not being enabled for EXAONE 4.5 ( #26848 ) model : fix SWA not being enabled for EXAONE 4.5 load_arch_hparams tests hparams.n_layer() == 64 before LLM_KV_NEXTN_PREDICT_LAYERS has been read. n_layer() returns n_layer_all - n_layer_nextn and n_layer_nextn defaults…
16 -
b10360
common/peg : suppress incomplete escape sequences ( #26780 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
25 -
b10359
ggml-webgpu: fix CI errors from #25025 and #25262 ( #26566 ) test new flash_attn test rebase and fix to disable subgrou matrices when max_kv_tile == 0 delete log output Add i32 support to cpy and enables the all ops test restore the non target ci tests comment out of TODO of…
26 -
b10358
Address review comment of PR 25532 ( #26852 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
35 -
b10357
opencl: transpose the K tile in local memory for FA prefill kernels ( #26428 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
20 -
b10362: tests : disable backend sampler hip multi output (#26878)
test-backend-sampler: skip multi_output_sampling_chain on HIP The new multi_output_sampling_chain test uses top_k, whose backend probs path needs CUB (unavailable on HIP), so sampled_probs is null and the test aborts. Add it to the existing HIP skip list alongside the other…
29 -
b10356
ci : target ROCm 7.14 for build and release ( #25775 ) Switch ROCm from 7.2.1 to 7.14 ROCm 7.14 is the first production release using TheRock build system. It can be installed using multi-arch deliverables from wheels, debs, rpms, tarballs or runfiles. Adjust ROCm targets for…
18 -
b10355
llama : support multi-output backend sampling ( #25532 ) Enable backend sampling with token speculation Clamp the mask sum before converting it into the sampled index Add a numeric context parameter declaring the maximum outputs one sequence More fixes Don't reuse memory for…
8 -
b10354
ggml-cpu : fix CPU affinity mask being ignored on Android ( #26838 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
37 -
b10353
ggml : require contiguous src for ROLL on CUDA and Metal ( #25928 ) ggml_roll only asserts nb[0] == ggml_type_size, so a permuted src is a valid input, but the CUDA and Metal roll kernels index by ne alone and never read the nb strides. A non-contiguous src therefore produced…
25 -
b10344
model: add MTP support for Nemotron model ( #26725 ) model: add MTP support for Nemotron Nano model model: add mtp_flags for nemotron model address review comments Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
23 -
b10343
vendor : update cpp-httplib to 0.53.0 ( #26821 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
5 -
b10342
model : Granite-Switch Architecture ( #25107 ) granite-switch: add llama.cpp backend (POC, CPU) New "granite-switch" architecture: a dense, all-attention Granite-4.1 model with N embedded LoRA adapters selected per-token by control tokens. gguf-py schema (arch, KV keys, stacked…
22 -
b10338
model-saver : fix expert shared/chunk FFN length key clobber ( #26693 ) The saver called add_kv with LLM_KV_EXPERT_SHARED_FEED_FORWARD_LENGTH twice, the second time passing n_ff_chexp. gguf_set_val_u32 removes-then-appends, so the second call clobbers the first: the saved…
5 -
b10336
ggml-webgpu : refactor several wgsl files and simplify flash_attn wgsl. ( #26134 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
15 -
b10335
ui: degrade the working directory picker when file search is off ( #26 …
18 -
b10333
ggml-cpu : fix missing Q5_0 dispatch in SpaceMiT backend ( #26792 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
4 -
b10332
ci: rm GGML_HIP_ROCWMMA_FATTN ( #26760 ) Signed-off-by: Aaron Teo [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
10 -
b10331
server: report the isolate working directory from get_info ( #26773 ) server: report the isolate working directory from get_info Without an explicit cwd, get_info fell back to the server process working directory even when a tools runtime was configured. That named a host path…
19 -
b10330
CUDA: fuse rms_norm + mul + rope (+ view + set_rows) ( #26767 ) CUDA: fuse rms_norm + mul + rope (+ view + set_rows) tests: add broadcast weight case to rms_norm_mul_rope CUDA: check memory ranges before rms_norm rope fusion CUDA: check memory ranges in rope set_rows fusion…
25 -
b10329
server, ui: only offer a working directory when a tool reads it ( #26762 ) The working directory chip showed up as soon as the server exposed any builtin tool, so a server started with just get_datetime, or a user who turned every filesystem tool off in the settings, still got a…
8 -
b10328
server: add initial tool isolation support (via docker) ( #26507 ) server: add initial tool isolation support (via docker) add docs adapt get_info py: fix type check cont separate tools_io_sandbox / tools_io_docker rename sandbox --> isolate x-tool-docker --> x-tool-runtime…
15 -
b10327
CUDA: fix thread/block count in quantized cpy kernel launches ( #26731 ) CUDA: fix thread/block count in quantized cpy kernel launches tests: add uneven block count cpy case Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
31 -
b10326
tts: account for the vocoder pass in the timings line ( #26733 ) get_output runs the waveform work the pipeline defers to it, from a single trailing window to a full pass depending on the model. Measuring it keeps the reported total and the audio to process ratio honest.…
6 -
b10322
sycl: coalesce the ssm_conv window loads ( #26612 ) test-backend-ops perf -o SSM_CONV on an Arc Pro B70, interleaved A/B against master, 6 reps, us/run: ne_a=[515,3328,1,1] ne_b=[4,3328,1,1] n_t=512 97.68 -> 52.95 1.85x ne_a=[937,8192,1,1] ne_b=[4,8192,1,1] n_t=934 516.16 ->…
33 -
b10321
metal : fix NORM/RMS_NORM for row lengths that leave a partial simdgroup ( #26708 ) ggml_metal_op_norm sized the threadgroup with nth = std::min(nth, args.ne00_t) , which can leave nth not a multiple of the simdgroup size. The kernels finish their row reduction with a…
7 -
b10319
mtmd: fix longest_edge ignoring min/max pixels ( #26638 ) mtmd: fix longest_edge ignoring min/max pixels nits Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
32 -
b10318
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
13