Many medical segmentation papers call K-fold CV ensembles “deep ensembles,” but they are not the same and give different uncertainty signals. True deep ensembles are better for reliable failure detection/calibration, while CV ensembles may better capture annotator ambiguity.</p>\n","updatedAt":"2026-05-21T16:24:54.605Z","author":{"_id":"65fb0a0d00bd001854095acb","avatarUrl":"/avatars/9108a47ee73c542102dc59e31b5bc5e1.svg","fullname":"Tristan","name":"Kirscher","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9067006707191467},"editors":["Kirscher"],"editorAvatarUrls":["/avatars/9108a47ee73c542102dc59e31b5bc5e1.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.18329","authors":[{"_id":"6a0d7d430cc88a0d483d375b","user":{"_id":"65fb0a0d00bd001854095acb","avatarUrl":"/avatars/9108a47ee73c542102dc59e31b5bc5e1.svg","isPro":false,"fullname":"Tristan","user":"Kirscher","type":"user","name":"Kirscher"},"name":"Kirscher Tristan","status":"claimed_verified","statusLastChangedAt":"2026-05-20T17:09:50.707Z","hidden":false},{"_id":"6a0d7d430cc88a0d483d375c","name":"Bujotzek Markus","hidden":false},{"_id":"6a0d7d430cc88a0d483d375d","name":"Kirchhoff Yannick","hidden":false},{"_id":"6a0d7d430cc88a0d483d375e","name":"Rokuss Maximilian","hidden":false},{"_id":"6a0d7d430cc88a0d483d375f","name":"Isensee Fabian","hidden":false},{"_id":"6a0d7d430cc88a0d483d3760","name":"Kahl Kim-Celine","hidden":false},{"_id":"6a0d7d430cc88a0d483d3761","name":"Kovacs Balint","hidden":false},{"_id":"6a0d7d430cc88a0d483d3762","name":"Maier-Hein Klaus","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/65fb0a0d00bd001854095acb/7d5wkv2pH9rk6hF_rbhTt.png"],"publishedAt":"2026-05-18T00:00:00.000Z","submittedOnDailyAt":"2026-05-21T00:00:00.000Z","title":"Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation","submittedOnDailyBy":{"_id":"65fb0a0d00bd001854095acb","avatarUrl":"/avatars/9108a47ee73c542102dc59e31b5bc5e1.svg","isPro":false,"fullname":"Tristan","user":"Kirscher","type":"user","name":"Kirscher"},"summary":"Ensemble disagreement is widely used as a proxy for epistemic uncertainty in medical image segmentation. In practice, many studies form ensembles via K-fold cross-validation (CV), yet refer to them as ``deep ensembles'' (DE). Because CV members are trained on different data subsets, their disagreement mixes seed-driven variability with data-exposure effects, which can change how uncertainty should be interpreted. We audit recent segmentation uncertainty studies and find that terminology--implementation mismatches are common. We then compare a standard 5-fold CV ensemble to a 5-member DE (fixed training set, different random seeds) under otherwise identical configurations on three multi-rater segmentation datasets spanning three modalities. We evaluate uncertainty for calibration, failure detection, ambiguity modeling, and robustness under distribution shift. DE match segmentation accuracy while improving calibration and failure detection, whereas CV ensembles sometimes correlate more strongly with inter-rater variability on the studied datasets. Thus, ensemble construction should be chosen to match the research question: DE for reliability-oriented use (e.g., selective referral/failure detection) and CV ensembles as a proxy for ambiguity. We provide a lightweight nnU-Net modification enabling DE training within the default pipeline.","upvotes":1,"discussionId":"6a0d7d440cc88a0d483d3763","githubRepo":"https://github.com/Kirscher/LostInFolds","githubRepoAddedBy":"user","ai_summary":"Deep ensembles trained with fixed data and varying seeds outperform cross-validation ensembles in calibration and failure detection for medical image segmentation, while cross-validation ensembles better approximate inter-rater variability.","ai_keywords":["deep ensembles","cross-validation","epistemic uncertainty","calibration","failure detection","ambiguity modeling","distribution shift","nnU-Net"],"githubStars":1,"organization":{"_id":"6819c17a86b01862017668af","name":"MIC-DKFZ","fullname":"MIC at DKFZ","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/67d0461f48276ed25e08047b/G6otR2N9xql06E3EJyOd4.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"65fb0a0d00bd001854095acb","avatarUrl":"/avatars/9108a47ee73c542102dc59e31b5bc5e1.svg","isPro":false,"fullname":"Tristan","user":"Kirscher","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6819c17a86b01862017668af","name":"MIC-DKFZ","fullname":"MIC at DKFZ","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/67d0461f48276ed25e08047b/G6otR2N9xql06E3EJyOd4.png"}}">
Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation
Abstract
Deep ensembles trained with fixed data and varying seeds outperform cross-validation ensembles in calibration and failure detection for medical image segmentation, while cross-validation ensembles better approximate inter-rater variability.
AI-generated summary
Ensemble disagreement is widely used as a proxy for epistemic uncertainty in medical image segmentation. In practice, many studies form ensembles via K-fold cross-validation (CV), yet refer to them as ``deep ensembles'' (DE). Because CV members are trained on different data subsets, their disagreement mixes seed-driven variability with data-exposure effects, which can change how uncertainty should be interpreted. We audit recent segmentation uncertainty studies and find that terminology--implementation mismatches are common. We then compare a standard 5-fold CV ensemble to a 5-member DE (fixed training set, different random seeds) under otherwise identical configurations on three multi-rater segmentation datasets spanning three modalities. We evaluate uncertainty for calibration, failure detection, ambiguity modeling, and robustness under distribution shift. DE match segmentation accuracy while improving calibration and failure detection, whereas CV ensembles sometimes correlate more strongly with inter-rater variability on the studied datasets. Thus, ensemble construction should be chosen to match the research question: DE for reliability-oriented use (e.g., selective referral/failure detection) and CV ensembles as a proxy for ambiguity. We provide a lightweight nnU-Net modification enabling DE training within the default pipeline.
Community
Many medical segmentation papers call K-fold CV ensembles “deep ensembles,” but they are not the same and give different uncertainty signals. True deep ensembles are better for reliable failure detection/calibration, while CV ensembles may better capture annotator ambiguity.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.18329 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.18329 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.18329 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.