[R] What 1000+ Harness Experiments Taught Me About Self-Improving Agents [R]
Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.
I recently wanted to see whether an AI agent could self-improve a harness to solve terminal bench tasks. It’s possible for an AI agent to propose a meaningful one-time change to the harness, but after experimenting with this for a couple of weeks, I think the continuous self-improvement is mostly an experiment-systems problem. The system needs a way to decide what kind of improvements can safely compound.
Turns out there's a lot of parallels to coding-agent customization (e.g. SKILLS.md etc..) too.
I wrote my experience of building such system here, including the successful and failure attempts during the process, and how I approached the self-improvement loop. It's not intended as a benchmark claim but more of a systems/research writeup.
https://www.henrypan.com/blog/2026-05-25-self-improvement-harness/
[link] [comments]
More from r/MachineLearning
-
noisekit - CLI for generating realistic degraded speech datasets for ASR benchmarking [P]
May 27
-
EMA-Gated Temporal Sequence Compression in Vision Transformers [P]
May 27
-
Cross-species RSA: same learning rules (BP, PC, STDP, FA) tested against both human fMRI and macaque electrophysiology [P]
May 27
-
Profiling PyTorch training without accidentally stalling the GPU [D]
May 27
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.