Prismix
r/LocalLLaMA · · 1 min read

LongCat-Flash-Lite-Sparse Is Now Available for Download

#model-release #long-context

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

Like Read original ↗
LongCat-Flash-Lite-Sparse Is Now Available for Download

The weights have now been added to the repo an hour ago.

This model is built upon LongCat-Flash-Lite, the differences are that LongCat-Flash-Lite-Sparse:

  1. Replaces dense MLA with LongCat Sparse Attention (LSA)

  2. Natively supports context lengths of up to 1M tokens (vs 256k for LongCat-Flash-Lite)

submitted by /u/LLMFan46
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA