Any tools to turn a codebase into a fine tuning dataset? [D]
Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.
I have a few web projects with pretty good UI/UX and I’m wondering if there’s any tool or workflow that can turn an existing codebase into a dataset for fine tuning.
For example, given a React/Next.js project with components, pages, styling, etc. or a static html site, I’d like to turn it into something like:
instruction/prompt -> code
or whatever format actually makes sense for training an instruct/thinking/diffusion coding model.
Also curious how people handle things like:
- keeping the context between components/files
- screenshots + code
- generating useful instructions instead of generic descriptions
I’m also working on a different model architecture that I think could improve quality/speed while using less VRAM, so I want to build a decent dataset and benchmark to test it properly.
Has anyone done something like this? Any tools, repos, papers, or workflows you’d recommend?
[link] [comments]
More from r/MachineLearning
-
Talus: a 23M-parameter diffusion model for game terrain, evaluated against a real-vs-real noise floor, running in the browser on WebGPU [P]
Oct 9
-
Integrum - Reflection based MCP server from any Python Module/Library [P]
Oct 9
-
Clarification ARR Commitment [D]
Oct 9
-
I built ALHR: A tree based sparse attention system that achieves sub-quadratic inference while retaining accuracy. [P]
Oct 9
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.