Clearbar

Verdict library

Real verdicts on public datasets. Check every number.

Each verdict runs the same examples through 8 models, counts every retry in the price and names the cheapest one that matches the baseline. The dataset, the prompt and a re-run script are on the page.

Classification · 150 public examples

meta-llama/llama-3.3-70b-instruct

Matches openai/gpt-4.1 on Banking customer-message intent classification (77 intents), at a fraction of the price.

$0.000082per successful task
93.4%under openai/gpt-4.1 ($0.001235)
77.3%pass rate vs 76.7% for the baseline

See the numbers and failures

JSON extraction · 150 public examples

mistralai/mistral-small-3.2-24b-instruct

Matches openai/gpt-4.1 on Structured JSON extraction from messy documents, at a fraction of the price.

$0.000203per successful task
96.1%under openai/gpt-4.1 ($0.005173)
62.7%pass rate vs 62.0% for the baseline

See the numbers and failures

What changed

How to read a verdict

  1. Pass rate on the dataset, with a 95% interval.
  2. Cost per successful task, retries and failed attempts included.
  3. Paired match against the baseline on the same tasks.
  4. Failures you can read, and the script to re-run them.

Get told when a verdict changes

Prices and models move every month. Leave your email and we'll send a note when a verdict here changes. One short email per change. Or follow the feed.

Used only for these notes. Privacy

Get this verdict on your own examples

Bring 50-300 input/expected pairs and your OpenRouter key. You get one named model, its cost per successful task and the failures it will have.

Pro run

$19 per run

  • Up to 150 of your examples
  • Up to 8 models plus your current one as the baseline
  • Verdict with cost per successful task, retry tax, failures and payback

We'll email you the day it opens. No payment now. Privacy

Deep run

$49 per run

  • Up to 300 of your examples
  • Up to 15 models plus your current one
  • 3 re-checks within 30 days when prices or models move

We'll email you the day it opens. No payment now. Privacy

Model watch

$9 per month

  • Your finalists re-checked when OpenRouter prices or models change
  • A dated change list: who got cheaper, who now clears your bar
  • Cached results reused, so only new or repriced cells run

We'll email you the day it opens. No payment now. Privacy