Title:MORSE: Multi-Context Ordering via Reverse Scoring for Evidence-Preserving Compression

Abstract:Likelihood-based context compression can account for cross-context redundancy through sequential scoring, but this makes compression outcomes sensitive to context order. We show that different permutations of the same context collection can produce markedly different evidence-retention outcomes under an unchanged compressor. We attribute this sensitivity to information preemption: earlier partially relevant contexts can absorb credit for shared information, suppressing the incremental score of later, stronger evidence carriers and increasing their risk of removal. Controlled pair-swap interventions directly support this mechanism by showing that evidence-first ordering substantially improves supporting-evidence survival. To address this problem, we introduce MORSE, a compression-aware method for evidence-preserving context ordering. MORSE applies a common reverse query-evidence principle to both individual contexts and compressed candidate outputs, using the former to construct an evidence-first anchor and the latter to guide compression-aware permutation selection. Across multi-hop QA benchmarks, compression procedures, budgets, and scoring models, MORSE consistently improves evidence preservation over static reverse ordering and compute-matched random search, with corresponding overall improvements in downstream QA. Our code is available at this https URL.

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools