r/MachineLearning · · 1 min read

[P] have a couple technical questions for my LLM router. [P]

Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.

I am a CS undergrad and I think token economics is the next big problem for companies. I am building a LLM router specifically for code and codebases. The Routing is not actually done by a heavily fine tuned llm(already existing solutions do this). Using a bit of a different approach. I am gauging the complexity by measuring interaction between signals that can be cheaply extracted from the prompt. One of these signals is what I like to call blooms_intent, based on bloom’s taxonomy. Bloom's taxonomy is a framework for categorizing educational goals. If a query is “What is this” it falls under remember category whereas “implement this” is more of create category.

Questions:-
How do I find datasets for this purpose.
Is bootstrapping datasets using AI fine for this. Should I do centroid based classification which I’ve been doing till now but the confidence difference between categories for ambiguous queries is way too close.
What is the best dataset size and classifier that can somewhat reliably differentiate nuances between queries.

You may ask why not use AI for these questions. I have and that’s why I’ve come here. Please lmk your thoughts and thanks in advance!!

submitted by /u/getridofaks
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/MachineLearning