Route uncertain Jev decisions to review

Validate a Jev Choice answer, route eligible labels and send uncertain or invalid answers to review. Includes executable policy tests; no model benchmark is claimed.

// Routing policy only: no API call, no model-quality claim.
// 0.85 is illustrative; choose a threshold on held-out labeled data.
export function routeAnswer(answer, threshold = 0.85) {
  if (!Number.isFinite(threshold) || threshold < 0 || threshold > 1)
    throw new RangeError('Threshold must be between 0 and 1');
  const allowed = ['quote', 'schedule', 'complaint', 'other'];
  if (!answer || !allowed.includes(answer.choice) ||
      !Number.isFinite(answer.confidence) ||
      answer.confidence < 0 || answer.confidence > 1)
    return { action: 'review', reason: 'invalid-answer' };
  if (answer.choice === 'other' || answer.confidence < threshold)
    return { action: 'review', reason: 'uncertain-or-other' };
  return { action: 'route', label: answer.choice };
}

import math

def route_answer(answer, threshold=0.85):
    if isinstance(threshold, bool) or not isinstance(threshold, (int, float)) or not math.isfinite(threshold) or not 0 <= threshold <= 1:
        raise ValueError("Threshold must be between 0 and 1")
    allowed = {"quote", "schedule", "complaint", "other"}
    if not isinstance(answer, dict):
        return {"action": "review", "reason": "invalid-answer"}
    choice, confidence = answer.get("choice"), answer.get("confidence")
    if not isinstance(choice, str) or choice not in allowed or isinstance(confidence, bool) or not isinstance(confidence, (int, float)) or not math.isfinite(confidence) or not 0 <= confidence <= 1:
        return {"action": "review", "reason": "invalid-answer"}
    if choice == "other" or confidence < threshold:
        return {"action": "review", "reason": "uncertain-or-other"}
    return {"action": "route", "label": choice}

Connect the documented Jev response

The official quickstart returns a Choice result under answers.intent when the question is named intent. Pass that object to routeAnswer(response.answers?.intent). Handle request failures by sending the case to review; do not invent a successful answer.

Read the confidence documentation before setting thresholds. Confidence is not an accuracy guarantee, and different tasks need different acceptance criteria.

What was actually tested?

On 24 September 2026, the JavaScript and Python policies each passed 12 synthetic routing fixtures and 3 invalid-threshold checks. Cases cover the threshold boundary, low confidence, the other label, missing fields, unknown labels, nonnumeric confidence and out-of-range values. These tests exercise our branching code only. No model API was called; model accuracy, calibration, latency and cost remain untested.

Download the JavaScript policy and JavaScript tests, or the Python policy and Python tests. Run node route-policy.test.mjs or python3 test_route_policy.py with each pair in the same folder.

What happens in a sample case?

A synthetic answer with choice quote and confidence 0.85 routes to quote. At 0.849 it goes to review. The other label goes to review even at confidence 1. These are policy expectations, not observed model predictions.

Notes

• The example threshold of 0.85 is illustrative, not a recommended production setting. Select it using labeled development data.
• Connect the review action to your own human queue or separately evaluated fallback. This example does not call a frontier model.
• Track wrong automatic routes, review rate, latency and total workflow cost. Do not log customer content without an appropriate data policy.