BitCPM-CANN: Native 1.58-Bit Large Language Model Training on Ascend NPU

Paper: https://github.com/OpenBMB/MiniCPM/blob/main/docs/BitCPM_CANN.pdf

Abstract

We present BitCPM-CANN, a systematic family-level study of 1.58-bit (ternary) quantization-aware training (QAT) on the Huawei Ascend NPU platform. To address two practical gaps for extreme low-bit LLMs—whether ternary weights preserve capabili- ties on complex reasoning tasks at on-device scales, and how to make end-to-end 1.58-bit training natively available outside the CUDA ecosystem—we port our prior GPU-based pipeline to CANN, MindSpeed, and Megatron-LM, and train four models (BitCPM- CANN-0.5B/1B/3B/8B) strictly aligned with their full-precision MiniCPM4 counterparts in architecture and pre-training data. Across 11 benchmarks spanning commonsense reasoning, domain knowledge, and mathematics & reasoning, the 1B, 3B, and 8B variants retain 95.7%–97.2% of full-precision performance, with the 3B variant achieving parity on BBH and the 3B/8B variants recovering nearly all of GSM8K. The 0.5B variant retains 90.1%, with the residual gap concentrated on mathematics, indicating that capacity—not the quantizer—is the bottleneck at sub-billion scales. Our QAT integration adds only a 4.5% training throughput overhead (148 vs. 155 TFLOP/s per NPU), making ternary training viable as a default configuration, while enabling up to an 8× weight memory reduction (approximately 6× end-to-end including scaling factors) at inference. To our knowledge, this is the first end-to-end 1.58-bit training system on a domestic NPU scaled up to 8B parameters, providing a reusable low-bit training infrastructure for the Ascend ecosystem

BitCPM-CANN was trained in ternary from scratch with the same data as MiniCPM4. MiniCPM4 8B achieves comparable performance with Qwen3-8B trained with 36 trillion tokens using only 8 trillion tokens. (MiniCPM4 was released last year: https://arxiv.org/abs/2506.07900)

• https://github.com/OpenBMB/MiniCPM

• https://huggingface.co/collections/openbmb/bitcpm-cann