---
name: openrouter-cost-optimizer
description: Pick a cost-effective OpenRouter model for a bulk LLM job by reading today's live usage rankings and live pricing, costing the specific job in dollars, and presenting the tradeoff for the user to choose. Use this BEFORE running any LLM job over more than a few hundred items — classification, labelling, extraction, enrichment, scoring, summarisation, translation, or anything that loops a model over rows. Use it whenever the user mentions OpenRouter, model choice, "which model should I use", API cost, token spend, a budget or credit limit, "make it cheaper", "too expensive", or is about to label, score or extract at volume. Use it even when a model is already chosen and the task sounds routine — a from-memory shortlist skips the open-weight and Chinese labs (DeepSeek, Qwen, Kimi, GLM, MiniMax, Tencent) that are routinely 5-20x cheaper for the same job, which is the single most common way an LLM pipeline silently overspends. Prefer this over guessing a model id or reusing whichever model the last script happened to use.
---

# OpenRouter cost optimizer — read the live board, cost the job, let the human choose

## Onboarding — start here

**What this does.** Turns "which model should I run this on?" into a decision with
numbers attached. It reads today's OpenRouter usage rankings and today's prices,
computes what *your specific job* costs on each candidate, and puts 2–4 real
options in front of you with the saving and the risk stated. You choose; it does
not choose for you.

**What it does not do, and where to go instead.**

- It does not tell you whether a cheap model is *good enough* for your task. No
  ranking can. It gives you a cheap way to find out — the 20-row bake-off in
  Step 4 — but the judgement stays with you.
- It does not optimise prompts for quality, only for size. For prompt engineering
  use the model docs; for Anthropic-specific behaviour use the `claude-api` skill.
- It does not manage keys, retries, rate limits or batching. That belongs in the
  job script.
- It is not a benchmark. Usage ranking says a model is *paid for at scale*, not
  that it is accurate on your data.

**Setup.** Needs `OPENROUTER_API_KEY` in the environment for pricing, and
`FIRECRAWL_API_KEY` for the rankings scrape. Rankings are optional — without
Firecrawl you still get live pricing and the job cost, just no usage signal.

```bash
set -a && . ./local.env && set +a
```

**Verify** with a one-line run. If you see a table with a `rank` column
populated, both halves are working:

```bash
./.venv/bin/python .agents/skills/openrouter-cost-optimizer/scripts/rank_models.py \
  --in-tokens 1000 --out-tokens 200 --rows 100
```

**How to invoke.** "Which model should I use for X?", "make this cheaper", "cost
this job before I run it", "I'm about to label 16,000 headlines".

**Cost per run.** Effectively free — one pricing API call and one Firecrawl
scrape (1 credit, ~$0.001). It is always cheaper to run this than to guess wrong
once.

**Rest of the skill.** `scripts/rank_models.py` does the fetching and the
arithmetic. `references/model-notes.md` holds what the rankings cannot tell you:
which labs to reach for by task shape, and the failure modes of the cheap tier.

---

## Why this exists

A real run in this repo: **$21.50 of OpenRouter spend where roughly half was
avoidable.** Not through anything clever — three ordinary mistakes:

| Mistake | Cost | Fix |
|---|---|---|
| Second model asked for a full record when only 9 enum fields were read | ~90% of its output tokens | Scope the prompt to what you consume |
| Raw page markdown sent in, including SVG paths and CDN image URLs | 55% of input tokens | Clean the input |
| Model chosen from memory, frontier tier by reflex | 2–20x | Read the live board |

Note the order. **Model choice is the last lever, not the first.** Halving the
tokens helps on every model; switching models helps only until you switch again.
Work through all three.

---

## The method

### Step 1 — Always read the live board first, and name the task

**Never pick from memory, and never skip straight to price.** Start every run by
fetching `https://openrouter.ai/rankings` — the script does this for you — and
identify which **task** the job is. The rankings page carries a "Top models by
task" treemap: each task's leading models ranked by *share of spend on OpenRouter*.

This matters because the overall usage chart and the per-task board disagree, and
the per-task board is the one you want. The headline chart is dominated by
whatever is cheapest at enormous volume; the task board shows where people
actually spend money for a specific kind of work. A real reading:

```
overall #1        deepseek/deepseek-v4-flash-0731
Classification    leading lab: openai
Data Extraction   leading lab: anthropic
Summarization     leading lab: google
Tool Dispatch     leading lab: moonshotai (Kimi)
Memory Extraction leading lab: z-ai (GLM)
Debugging         leading lab: z-ai (GLM)
```

Three different tasks, three different labs, none of them the overall leader.
Picking the top of the headline chart for a data-extraction job is the same
mistake as picking from memory, just with a fresher number.

Match the job to a task before going further:

| Your job | Task to pass |
|---|---|
| Labelling headlines, tagging rows, routing | `Classification` |
| Pulling fields out of scraped pages or documents | `Data Extraction` |
| Reshaping records, normalising, mapping schemas | `Data Transformation` |
| Condensing long text | `Summarization` |
| Deciding which tool or branch to take | `Tool Dispatch` |
| Drafting prose | `Content Writing` |

Run without `--task` to print the full list from the live board.

### Step 2 — Get the real token profile, do not estimate it

The single biggest source of a wrong cost estimate is guessing input size. Measure
it on actual data:

```python
sample = open("one_real_input.txt").read()
in_tokens = len(sample) / 4          # ~4 chars per token, close enough to decide
```

For output, take what the job actually needs to return. A classifier emitting a
label and a confidence is ~30 tokens; a structured record with prose is 2,000+.
If you are unsure, run one item and count.

**Reasoning models bill their thinking as output.** If a candidate is a reasoning
model, its real output can be several times what you asked for — which is why a
measured estimate can still come in under the invoice. When comparing, prefer a
non-reasoning model for mechanical work; you are not paying for thought you do
not need.

### Step 3 — Run the ranker with the task

```bash
./.venv/bin/python .agents/skills/openrouter-cost-optimizer/scripts/rank_models.py \
  --task "Data Extraction" --in-tokens <measured> --out-tokens <measured> --rows <N>
```

You get every priced model costed for *this* job, with today's usage rank, the
lab, and a `*` on models from the lab leading your task. It also proposes three
bets, which are the shape the choice should take:

- **BEST** — the task-leading lab's most-used model. The capability bet.
- **MID** — a balance point, usually a flash/mini tier.
- **CHEAPEST** — the price floor that still carries a real usage signal.

Add `--json` to feed the numbers straight into a question.

### Step 4 — Present the bets with AskUserQuestion, and never auto-pick

Cost is objective. "Good enough for this task" is a judgement about the user's
data and their tolerance for error, and they own it. Spending their money on the
cheapest row without asking is the same mistake as spending it on the most
expensive one by reflex.

Build the options so they are genuinely different bets, not four flavours of the
same one:

1. **BEST** — the task-leading lab. Justified when a wrong row is expensive.
2. **MID** — the balance point. The safe default for anything with nuance.
3. **CHEAPEST** — usually an open-weight or Chinese-lab model. Justified when
   the work is mechanical and the answer is literally present in the input.
4. **The current model, priced** — so the saving is a number rather than a
   promise. Include it even when it is obviously worst; it is the baseline the
   user is deciding against.
4. *(when it applies)* **Batch endpoints** — the `:batch` suffix is ~50% off the
   same model, paid for in latency (async, minutes to hours). Free money for an
   overnight job; useless when someone is waiting on the result.

State for each: **dollar cost for this job**, the multiple against the baseline,
and the honest risk. Do not editorialise the cheap option into sounding reckless
or the expensive one into sounding safe — put the numbers down and let them pick.

### Step 5 — Bake off on 20 rows before committing to 16,000

This is the step that makes choosing a cheap model rational rather than hopeful,
and it costs cents.

Run the top candidate and the incumbent over the same 20 real items. Compare on
whatever the job's actual success criterion is — exact-match label, valid JSON,
field-level agreement. Then decide with evidence.

```
agreement >= 95%   ship the cheap model
85-95%             sample 100 more, or use the cheap model with the expensive one as tiebreak
< 85%              the saving is not real; a wrong label costs more than the tokens saved
```

An 80%-accurate model at one-tenth the price is not a bargain when the output
feeds a decision. Say that plainly if the numbers come out that way.

### Step 6 — Report what was actually spent

After the job, read the real figure rather than trusting the estimate:

```bash
curl -s https://openrouter.ai/api/v1/key -H "Authorization: Bearer $OPENROUTER_API_KEY"
```

If the invoice beat the estimate by more than ~30%, something in the token
profile was wrong — usually reasoning tokens or retries. Say so, and correct the
estimate for next time rather than quietly absorbing it.

---

## Where the cheap tier is genuinely fine, and where it is not

Ranking tells you a model is used. It cannot tell you it is right for your task.
The pattern that holds up:

**Cheap models do mechanical work well.** Classifying a short string into a fixed
set, extracting a field that is literally present, reformatting, translating,
deduplicating. The answer is *in* the input and the job is to find it.

**Cheap models degrade on judgement.** Weighing contradictory evidence, deciding
what a page is really selling, refusing to answer when the source is silent.
Notably they are worse at **leaving a field empty** — a small model asked for a
structured record will fill every slot rather than admit the text does not say.
If your pipeline depends on honest gaps, test that specifically.

**A cheap second opinion is often the right shape.** Where two models must agree,
the second only needs to be *independent*, not authoritative. Downgrading the
disagreement-checker is nearly free and costs nothing in signal.

See `references/model-notes.md` before picking for an unusual task shape.

---

## The three levers, in the order that pays

Run through these before accepting any quote, including this skill's own.

**1. Cut the input.** Scraped HTML and markdown are mostly not prose — image
markdown with CDN URLs, SVG path data, nav repeated on every page, tracking
query strings. Stripping it is lossless for most extraction tasks and routinely
removes half the tokens. Measure before and after; do not assume.

**2. Scope the output to what you read.** If the code consumes nine fields, ask
for nine fields. A model asked for a full record when you parse a fraction of it
is billing you for text nobody will ever see. This is the most commonly missed
lever because the waste is invisible in the code.

**3. Then choose the model.** With the job already half the size, the model
decision is smaller in absolute terms — which is the point. Do it last so you are
optimising the right number.

---

## Output format

End with a short, checkable summary — the user should be able to sanity-check the
arithmetic without rerunning anything:

```
Job:        16,000 headlines x (120 in + 60 out) = 2.9M tokens
Baseline:   anthropic/claude-haiku-4.5      $0.34
Chosen:     deepseek/deepseek-v4-flash-0731 $0.33   (rank #1 this week, 1.0x)
Bake-off:   19/20 agreement on real rows
Levers:     input already minimal; output capped at 30 tokens/row
```