Training data needs a real go/no-go gate before training [D]

We have gates for code, infrastructure, deployment and model performance.

But when it comes to the actual training artifact, the decision to proceed is often still spread across notebooks, validation scripts, dashboards and human judgment.

That feels like a weak point.

I’ve been thinking about what a proper pre-training control layer would look like.

Not another tool that just produces a long list of warnings, but a local system that audits the exact artifact about to be used and gives a reproducible PASS, WARNING, FAIL or FAIL_SECURITY verdict.

The decision would come from explicit evidence and hard gates: leakage, contradictions, redundancy, coverage, provenance, evidence integrity and how far the artifact is from its declared training objective.

The key difference is that an LLM would not decide the verdict. The same artifact, objective and configuration should produce the same result. A critical failure should not disappear inside a decent aggregate score.

It could also produce a repair plan, apply only approved changes to a derived copy, preserve the original and run a second audit afterward. Everything tied to manifests, checksums and the exact execution.

To me, this looks like a missing layer between data preparation and training.

But I can also see the strongest objection: training-data quality is contextual, and a formal verdict could create false confidence if the system is not extremely transparent.

So I’m curious how people working with real training pipelines would receive something like this.

Would you let it block a training run?

Would you trust the verdict, or only the evidence behind it?

And what would it need to prove before your team took it seriously?