Pyrecall open source tool for detecting catastrophic forgetting during LLM fine-tuning[P]
Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.
Surprised there's no real tooling for this given how much research exists on continual learning.
Built pyrecall to fill the gap. Snapshots skill scores before/after fine-tuning, flags regressions, rolls back LoRA adapters by name.
Fully local, no external APIs. v0.1.0, MIT, pip install pyrecall
Curious if anyone has thoughts on the benchmark design that's the part I'm least confident about.
[link] [comments]
More from r/MachineLearning
-
Derivative-Free Neural Network Optimization: MNIST Case [R]
Jun 13
-
is a preprint from an independent researcher worthy of arxiv endorsement if it got cited by a Peking University lab's paper 1 month after release? [D]
Jun 12
-
AMAZON ML SUMMER SCHOOL 2026 [D]
Jun 12
-
Just thinking, what about conducting a 1 day virtual session on fundamentals of computer vision ??? [D]
Jun 12
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.