Vet a claim with five typed questions
ChoiceNoulScoreStarter
Send a social post or article claim as state, ask who's making it, whether there's checkable evidence, whether its numbers show their method, whether comparisons are fair, and how promotional it reads. Your code turns the five answers into a verdict. This is the rubric behind every entry on this site.
// Request shape follows TypeSafe's launch docs. Confirm field names at docs.typesafe.ai.
const res = await fetch("https://api.typesafe.ai/v1/systemone", {
method: "POST",
headers: {
"Authorization": `Bearer ${process.env.TYPESAFE_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: "jev-1.13",
state: postText,
questions: {
source: { type: "choice", instructions: "Who is making this claim?",
criteria: {
vendor: "The company selling the product",
partner: "A paid or integration partner",
independent: "An unaffiliated user or reviewer",
} },
evidence: { type: "noul",
instructions: "Does the post link a repo, live demo, or dataset someone else could check?" },
false_precision: { type: "noul",
instructions: "Does it give exact numbers without showing how they were measured?" },
fair_comparison: { type: "noul",
instructions: "If it compares systems, were they measured under the same conditions?" },
hype: { type: "score", instructions: "How promotional versus factual is the language?",
min: 2, max: 10 },
},
}),
});
const { answers } = await res.json();
// Thresholds live in your code, not the prompt.
function verdict(a) {
if (a.evidence.noul < 0.3 && a.false_precision.noul > 0.6) return "unverified";
if (a.fair_comparison.noul < 0.3 && a.source.choice === "vendor") return "misleading";
if (a.evidence.noul > 0.8 && a.hype.score < 5) return "verified";
return "needs_review"; // send to a frontier model, then a person
}# Laya, running locally. Laya ships a Jev-compatible system_one(state, questions)
# helper; check the Laya README for the exact import in your installed version.
from laya import system_one
answers = system_one(
state=post_text,
questions={
"source": {"type": "choice", "instructions": "Who is making this claim?",
"criteria": {"vendor": "The company selling the product",
"partner": "A paid or integration partner",
"independent": "An unaffiliated user or reviewer"}},
"evidence": {"type": "noul",
"instructions": "Does the post link a repo, live demo, or dataset someone else could check?"},
"false_precision": {"type": "noul",
"instructions": "Does it give exact numbers without showing how they were measured?"},
"fair_comparison": {"type": "noul",
"instructions": "If it compares systems, were they measured under the same conditions?"},
"hype": {"type": "score", "instructions": "How promotional versus factual is the language?",
"min": 2, "max": 10},
},
)
# Laya's base checkpoint is weak zero-shot. Fine-tune on 200+ labeled posts
# and fit a temperature before trusting these probabilities.
API field names follow each model's launch documentation. Check TypeSafe's docs and the Laya README before shipping, since both are changing weekly.
Notes
- Jev works on this zero-shot. Laya needs fine-tuning first; expect near-chance answers from the base checkpoint.
- Keep the thresholds in code so you can tune them without touching the questions.
- Never let this decide alone that something is false. It estimates how much scrutiny a claim has survived.