Typesafe's JEV model work as an LLM [P]

I built a conversational AI that doesn't generate a single token — it selects from 400 pre-written responses using TypeSafe's Jev, a non-generative model that returns probabilistic judgments instead of text.

The technical approach:

Traditional LLMs generate responses token by token. I flipped this — I wrote 400 responses by hand and let the AI pick the best match. Here's the architecture:

Step 1: Classification (400ms, 1 API call)

I ask Jev 18 questions in parallel about the user's message:

Questions asked simultaneously: - intent (Choice: greeting/complaint/question/etc — 14 options) - formality (Score: 0-4 scale, 5 levels) - emotional_intensity (Score: 0-4 scale) - needs_human (Noul: yes/no probability) - subcategory_greeting (Choice: if intent=greeting, which type?) - subcategory_complaint (Choice: if intent=complaint, which type?) ... [14 speculative subcategory questions, one per intent] 

All 18 run in one API call using speculative fan-out — code only reads the subcategory answer for whichever intent wins. This eliminates a round trip without extra cost.

Step 2: Candidate Fetch (0ms, pure code)

The response bank is structured as:

{ complaint: { empathetic: [ { id: "cmp_emp_1", text: "Three times is way too many..." }, { id: "cmp_emp_2", text: "I hear you — that's frustrating..." }, // ...10 responses total ], solution_oriented: [ ... ] }, greeting: { ... } } 

Code looks up bank[intent][subcategory] and pulls ~10 candidates. If there are >10 in a category, I run a cheap Noul shortlist (one yes/no question per candidate) to narrow it down.

Step 3: Scoring (400ms, 1 API call)

Each candidate gets scored on 6 dimensions in parallel:

Per candidate (10 candidates × 6 dimensions = 60 questions): - relevance (Score: 0-4) - tone_match (Score: 0-4) - helpfulness (Score: 0-4) - answers_question (Noul: 0-1) - specificity (Score: 0-4) - natural_flow (Score: 0-4) 

Then I apply context-aware weights:

// Default weights weights = { relevance: 0.25, tone: 0.15, helpfulness: 0.20, ... } // High emotion? Tone matters more if (emotional_intensity > 2.5) { weights.tone = 0.30; weights.relevance = 0.20; } // Small talk? Natural flow matters more if (intent === "small_talk") { weights.natural_flow = 0.35; weights.helpfulness = 0.05; } 

Winner = highest weighted score.

Step 4: Output

If final score < 0.4 or intent confidence < 0.3, optional LLM fallback kicks in (I use Claude Haiku). Otherwise, return the selected response.

Example flow:

User: "I'm frustrated, my order has been wrong three times"

Classify (400ms): intent: complaint (91% confidence) subcategory: empathetic (speculative answer, used since complaint won) emotional_intensity: 3.8/4.0 formality: 1.2/4.0 needs_human: 0.34 (below 0.8 threshold, proceed) Fetch: 10 responses from complaint/empathetic Score (400ms): Each candidate × 6 dimensions Weights adjusted: tone=30% (emotion is high) Top 3: #1: "Three times is way too many..." → 0.82 #2: "I hear you — that's frustrating..." → 0.77 #3: "That sounds really frustrating..." → 0.70 Winner: Response #1 (highest weighted score) 

Total: ~800ms, ~$0.0001 per message

Why this works:

Philosophical questions nail it. Philosophy is about picking the right framing, not generating insights. "What's the purpose of life?" has many valid perspectives — selection feels natural.

No hallucinations. Every word was pre-written. Perfect for customer support or anywhere brand safety matters.

Full transparency. You see intent → scores → winner. No "the model decided" black box.

100x cheaper. ~$0.0001 vs ~$0.01-0.10 per message.

Where it breaks:

Can't generate anything. No coding, writing, creative tasks. If it's not in the bank, I got nothing.

Limited to 400 responses. Works for bounded conversations (support, FAQs, onboarding). Scaling needs manual work or template responses with dynamic slots.

Factual Q&A is rough unless you template responses with live data.

I added an LLM fallback for low confidence, but 90% is pure selection.

What Jev is NOT doing:

It doesn't read the response bank — I fetch candidates in code. Jev only classifies the message and scores the candidates. No reasoning, no generation, just fast probabilistic judgments.

Links:

• Live demo: https://jevllm.pages.dev (terminal mode + pipeline explorer)
• Source code: https://github.com/akash-kamat/jev-llm

Would love feedback — especially if anyone's tried similar selection-based approaches or has ideas for scaling the response bank!