<a href=\"https://github.com/GAIR-NLP/RIBOSPAN-FM\" rel=\"nofollow\">https://github.com/GAIR-NLP/RIBOSPAN-FM</a></p>\n","updatedAt":"2026-08-25T08:20:13.633Z","author":{"_id":"6540fdaa5d60444d17dc62c7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6540fdaa5d60444d17dc62c7/9QSHJhShxtZ81IqjXZk68.jpeg","fullname":"TangBohao(SII)","name":"Aphelios-Tang","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.8530396223068237},"editors":["Aphelios-Tang"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6540fdaa5d60444d17dc62c7/9QSHJhShxtZ81IqjXZk68.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.22849","authors":[{"_id":"6a8cffac8dd056518b7f54ad","user":{"_id":"67faa4fa7cfcb309de2b70a2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67faa4fa7cfcb309de2b70a2/w3441FzdAzJm5FoSbuO48.jpeg","isPro":false,"fullname":"Ziyuan Wang","user":"Ziyuan-Wang","type":"user","name":"Ziyuan-Wang"},"name":"Ziyuan Wang","status":"claimed_verified","statusLastChangedAt":"2026-08-25T08:14:05.531Z","hidden":false},{"_id":"6a8cffac8dd056518b7f54ae","user":{"_id":"6540fdaa5d60444d17dc62c7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6540fdaa5d60444d17dc62c7/9QSHJhShxtZ81IqjXZk68.jpeg","isPro":false,"fullname":"TangBohao(SII)","user":"Aphelios-Tang","type":"user","name":"Aphelios-Tang"},"name":"Bohao Tang","status":"claimed_verified","statusLastChangedAt":"2026-08-25T08:14:02.287Z","hidden":false},{"_id":"6a8cffac8dd056518b7f54af","name":"Fei Zhang","hidden":false},{"_id":"6a8cffac8dd056518b7f54b0","name":"Shuo Han","hidden":false},{"_id":"6a8cffac8dd056518b7f54b1","name":"Pengfei Liu","hidden":false}],"publishedAt":"2026-08-24T00:00:00.000Z","submittedOnDailyAt":"2026-08-25T00:00:00.000Z","title":"RIBOSPAN: A Long-Context RNA Foundation Model for Versatile RNA Modeling","submittedOnDailyBy":{"_id":"6540fdaa5d60444d17dc62c7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6540fdaa5d60444d17dc62c7/9QSHJhShxtZ81IqjXZk68.jpeg","isPro":false,"fullname":"TangBohao(SII)","user":"Aphelios-Tang","type":"user","name":"Aphelios-Tang"},"summary":"Full-length RNAs, particularly messenger RNAs, often exceed the context lengths used to pretrain existing RNA foundation models, limiting complete-transcript modeling at single-nucleotide resolution. We present RIBOSPAN, a 1.61-billion-parameter bidirectional RNA foundation model natively pretrained with context lengths up to 10,240 nt. RIBOSPAN combines dense bidirectional self-attention, single-nucleotide tokenization, and attention-isolated sequence packing to enable high-resolution modeling of complete long RNAs. We evaluate the model through nucleotide reconstruction, a controlled long-context representation benchmark, and frozen RNA-type representation analysis. Native 10K pretraining preserves strong reconstruction at 10,240 tokens, while continued pretraining with 40% masking improves recovery under heavy corruption while preserving representation quality. The long-context benchmark further shows that native 10K models maintain strong contextual responsiveness and context-specific representation separation while keeping perturbation-induced representation changes highly localized. Inference-time YaRN scaling recovers much of the contextual organization lost by direct extrapolation of short-context models, but induces substantially greater distal representation diffusion. Frozen-representation evaluations further demonstrate state-of-the-art RNA representation quality, with RIBOSPAN achieving the strongest overall performance across diverse RNA types and retaining a clear advantage on long RNAs. Building on the same backbone, we develop a multidimensionally conditioned discrete-diffusion framework for full-length mRNA generation and redesign, including synonymous-codon diffusion for protein-preserving CDS optimization. Together, RIBOSPAN establishes a powerful long-context foundation for transferable RNA representation learning and full-transcript mRNA design.","upvotes":2,"discussionId":"6a8cffac8dd056518b7f54b2","githubRepo":"https://github.com/GAIR-NLP/RIBOSPAN-FM","githubRepoAddedBy":"user","ai_summary":"RIBOSPAN is a large bidirectional RNA foundation model pretrained on up to 10,240 nucleotides that enables high-resolution full-transcript modeling, strong long-context representations, and discrete-diffusion-based mRNA generation and redesign.","ai_keywords":["bidirectional self-attention","single-nucleotide tokenization","sequence packing","YaRN scaling","discrete-diffusion","synonymous-codon diffusion","RNA foundation model","long-context representation"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"69c52037a0e053cf761732ed","name":"SII-GAIR-NLP","fullname":"SII-GAIR","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65029ac634ddd2032740dd82/WENFJEEOJIpWhzHNK_rWo.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6540fdaa5d60444d17dc62c7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6540fdaa5d60444d17dc62c7/9QSHJhShxtZ81IqjXZk68.jpeg","isPro":false,"fullname":"TangBohao(SII)","user":"Aphelios-Tang","type":"user"},{"_id":"67faa4fa7cfcb309de2b70a2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67faa4fa7cfcb309de2b70a2/w3441FzdAzJm5FoSbuO48.jpeg","isPro":false,"fullname":"Ziyuan Wang","user":"Ziyuan-Wang","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"69c52037a0e053cf761732ed","name":"SII-GAIR-NLP","fullname":"SII-GAIR","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65029ac634ddd2032740dd82/WENFJEEOJIpWhzHNK_rWo.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.22849.md","query":{}}">
RIBOSPAN: A Long-Context RNA Foundation Model for Versatile RNA Modeling
Abstract
RIBOSPAN is a large bidirectional RNA foundation model pretrained on up to 10,240 nucleotides that enables high-resolution full-transcript modeling, strong long-context representations, and discrete-diffusion-based mRNA generation and redesign.
Full-length RNAs, particularly messenger RNAs, often exceed the context lengths used to pretrain existing RNA foundation models, limiting complete-transcript modeling at single-nucleotide resolution. We present RIBOSPAN, a 1.61-billion-parameter bidirectional RNA foundation model natively pretrained with context lengths up to 10,240 nt. RIBOSPAN combines dense bidirectional self-attention, single-nucleotide tokenization, and attention-isolated sequence packing to enable high-resolution modeling of complete long RNAs. We evaluate the model through nucleotide reconstruction, a controlled long-context representation benchmark, and frozen RNA-type representation analysis. Native 10K pretraining preserves strong reconstruction at 10,240 tokens, while continued pretraining with 40% masking improves recovery under heavy corruption while preserving representation quality. The long-context benchmark further shows that native 10K models maintain strong contextual responsiveness and context-specific representation separation while keeping perturbation-induced representation changes highly localized. Inference-time YaRN scaling recovers much of the contextual organization lost by direct extrapolation of short-context models, but induces substantially greater distal representation diffusion. Frozen-representation evaluations further demonstrate state-of-the-art RNA representation quality, with RIBOSPAN achieving the strongest overall performance across diverse RNA types and retaining a clear advantage on long RNAs. Building on the same backbone, we develop a multidimensionally conditioned discrete-diffusion framework for full-length mRNA generation and redesign, including synonymous-codon diffusion for protein-preserving CDS optimization. Together, RIBOSPAN establishes a powerful long-context foundation for transferable RNA representation learning and full-transcript mRNA design.
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.22849 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.