llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 22d ago
b10103
metal : add f16 type support to leaky relu ( #25981 )
23 -
b10091
ci : fix SYCL package shared library lookup ( #25987 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
5 -
b10090
webgpu : add CONV_2D_DW (depthwise conv2d) kernel ( #25847 ) webgpu : add CONV_2D_DW (depthwise conv2d) kernel Implement GGML_OP_CONV_2D_DW for the WebGPU backend, ported from the Vulkan backend's conv2d_dw.comp. Assisted-by: Claude Opus-4.8 Remove unnecessary comments in webgpu…
37 -
b10089
cuda: GET_ROWS quants ( #25962 ) cuda: add k-quant support to GET_ROWS Device-side embedding lookups require GET_ROWS to handle the k-quants used by common GGUF recipes (Q4_K_M stores token_embd as q6_K). Without it the backend rejects the op and the scheduler falls back to the…
31 -
b10087
Add support for Laguna XS.2 & M.1 ( #25165 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
7 -
b10085
mtmd : use align_corners for qwen3vl vision position embedding interpolation ( #25781 ) The Qwen3-VL learned position embedding is interpolated to the runtime patch grid with the default bilinear+antialias (align_corners=False) sampling, while the transformers reference uses…
32 -
b10084
hexagon: check tensor type when reusing descriptors ( #25968 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
38 -
b10083
cuda: add sqrt_softplus in topk-moe for dsv4 ( #25896 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
33 -
b10082
kleidiai : warn once when a weight type has no KleidiAI kernel ( #25701 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu…
21 -
b10081
common: resolve draft repo to its requested sidecar ( #25955 ) With -hfd pointing to a repo shipping speculative sidecars, the draft resolved to the main model of that repo, since find_best_model() excludes sidecar files, and the explicit draft plan suppressed the sidecar…
26 -
b10080
server: return 400 instead of 500 on validation error with X-Conversation-Id ( #25760 ) server: return 400 instead of 500 on validation error with X-Conversation-Id set_req() attaches the spipe as soon as the header is present, before the request body is parsed. When params…
5 -
b10079
server : properly handle null llama_context ( #25868 ) Co-authored-by: Stanisław Szymczyk [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
11 -
b10078
vulkan: Refactor vk_queue to use per-instance mutexes and unique handles ( #23570 ) Refactor vk_queue to use per-instance mutexes and unique handles integrates VK_KHR_internally_synchronized_queues, abstracting the queue submission into a polymorphic interface that completely…
31 -
b10077
ggml-openvino: Add GGML_BACKEND_DL_IMPL invocation for OpenVINO backend ( #25795 ) This adds the missing GGML_BACKEND_DL_IMPL() macro invocation, that other backends have. Fixes #25586 for me Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
6 -
b10076
CUDA: vectorize same-type get_rows with int4 copy ( #25929 ) k_get_rows_float did a scalar one-element-per-thread copy and recomputed the row-invariant work (index load, fast_div_modulo, src/dst row pointers) for every element. Hoist that out of the per-element loop, and add a…
26 -
b10075
hexagon: add CLAMP op ( #25934 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
19 -
b10069
opencl: Support broadcast for Adreno MUL_MAT and honor view_offs for Adreno Q8_0 MUL_MAT for llama-server multi-stream ( #25910 ) opencl: handle broadcast for adreno gemm/gemv_noshuffle opencl: honor view_offs for adreno noshuffle gemm/gemv opencl: general GEMM/GEMV support…
14 -
b10068
model: rotate injected K/V cache for DFlash ( #25823 ) dflash: rotate injected K/V cache when using K/V quantization Update src/models/dflash.cpp Co-authored-by: Georgi Gerganov [email protected] clearer format remove trailing whitespace Co-authored-by: Georgi Gerganov…
21 -
b10067
llama-quant : exclude i32 ffn_gate_tid2eid routing table from quantization ( #25787 ) DeepSeek-V4's ffn_gate_tid2eid tensor is an i32 token-id -> expert-id index table, not weights. It was never added to the name-based exclusion list alongside ffn_gate_inp.weight, so…
20 -
b10066
opencl: load and use kernel_gemm_moe_q6_k_f32_ns from bin kernel lib ( #25797 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
19 -
b10064
opencl: transpose q4_K noshuffle scales for coalesced reads ( #25805 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu…
4 -
b10063
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
37 -
b10061
tests : initialize all tensors in test_dsv4_hc to avoid NaNs in sentinel tensors ( #25822 ) Co-authored-by: Stanisław Szymczyk [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
26 -
b10059
ggml-blas: default hadamard mul_mat to cpu routine ( #25710 ) Signed-off-by: Aaron Teo [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
8 -
b10058
vulkan: Support Q2_0 ( #25430 ) vulkan: Support Q2_0 The backend perf tests for mat-vec-mul weren't very good at first (worse than q2_k), doubling the rows per workgroup made a big difference. reorder resolve merge conflict, adjust err threshold for f16->q2_0 set_rows Website:…
37 -
b10057
sycl: fix row calculation when K_QUANTS_PER_ITERATION is 1 ( #25690 ) sycl: fix incorrect row calculation when K_QUANTS_PER_ITERATION=1 Signed-off-by: Todd Malsbary [email protected] sycl: use K_QUANTS_PER_ITERATION for non-reordered Q5_K kernel This is the only Q5_K…
24 -
b10054
docs: added a note about using OpenCl with Adreno 810 ( #25786 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
7 -
b10052
hexagon: L2 cache handling rework (dirty bit tracking with lazy flushing) and more MUL_MAT updates ( #25762 ) hex-mm: fix artificial limit in the solver that restricted number of act-prep threads hex-mm: fix warning hex-prof: do not apply --top to the timeline report hmx-mm: add…
18 -
b10051
kleidiai: Add SME vs SME2 distinction in kernel dispatch ( #25478 ) The current integration treats SME as a single capability (CPU_FEATURE_SME) with no distinction between SME(v1) and SME2. The kernels dispatched under CPU_FEATURE_SME use SME2-specific instructions, making…
10 -
b10050
vulkan: when using transfer queue for async copies, sync on event_wait to avoid race ( #25229 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
27 -
b10048
TP: fix Phi3, Bert, Plamo2/3, ChatGLM ( #25536 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
14 -
b10047
vendor: update BoringSSL to 0.20260713.0 ( #25624 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
24 -
b10046
tests: actually exercise test-recurrent-state-rollback ( #25758 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
32 -
b10045
server : allow text-only slot save/restore with mtmd ( #25076 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
25 -
b10043
CUDA: Support CUDA Virtual Devices ( #25228 ) support cuda virtual devices disable NCCL path when virtual devices are used label virtual devices in description; add GPUx2 server CI jobs code refactor Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
6 -
b10042
Enable CUDA graphs on volta+turing ( #25749 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
35 -
b10041
server: Ignore empty / non-existing Origin headers ( #25756 ) Otherwise this gives lots of unnecessary warnings: W srv operator(): (CORS) skip non-localhost origin: Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
8 -
b10040
ggml-cuda : restore prop.integrated on HIP builds ( #24233 ) PR #16308 set info.devices[id].integrated = false unconditionally for all CUDA/HIP devices as a workaround for corrupted output on Jetson Orin ( #15034 ). On HIP/ROCm the device's real hipDeviceProp_t.integrated flag…
37 -
b10038
ci : add official website link to release notes ( #25728 ) Assisted-by: pi:llama.cpp/Qwen3.6-27B Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
4 -
b10037
quant : allow using manual tensor types with --pure ( #25716 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
32 -
b10036
opencl: disable FA and MoE weights repack to work around compiler issues for Adreno 850 GPU ( #25745 ) opencl: workaround for A850 compiler compat opencl: fix DX compiler version parsing and cleanup Co-authored-by: Li He [email protected] macOS/iOS: macOS Apple Silicon…
7 -
b10035
cuda: extract Q1_0 elements via __byte_perm ( #25628 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64…
18 -
b10034
opencl: exclude some moe kernels on Adreno a7x ( #25698 ) opencl: exclude Adreno A7x from using Adreno MoE kernels Some compilers for A7x devices miscompile the repack kernels, corrupting the weights and causing MoE models to generate garbage output opencl: exclude A6x and…
22 -
b10032
cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) ( #25545 ) cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) chore : remove indentation of #pragma unroll cuda : remove unnecessary kernel template…
37 -
b10031
tokenize : drop --stdin mutual-exclusion check ( #25672 ) match cli and completion, which don't enforce it macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
13 -
b10025
cuda : relax tensor contiguity requirements for quantized concat ( #25678 ) cuda : relax tensor contiguity requirements for quantized concat tests : add test cases for non-contiguous quantized concat ggml : relax contiguity requirements for quantized concat Co-authored-by:…
22 -
b10021
DeepseekV4: reduce graph splits ( #25702 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan)…
9 -
b10020
sycl : fix get_rows Q2_K, Q4_K, Q5_K ( #25656 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan)…
35