OpenRouter catalog
Check models here, then run them on this tab. Scores land on Runs and
the public page.
Live calls need OPENROUTER_API_KEY in Railway or .env.
Loading catalog…
0 selected
Stratum occupancy
Curation principles
Writes a public scorecard without calling models.
Problem bank
Titles, skills, and named traps only. Prompt text is not loaded into the browser.
| Id | Field | Mode | Skill | Title | Trap |
|---|
Stratified subsample
| Public id | Field | Mode | Skill | Title |
|---|
Ingest a snapshot
Drop a Claude Code, claude.ai, or Cursor conversation (markdown, json, or jsonl).
For coding, put hidden tests in a ```python tests fence (or upload a grader).
The chat’s assistant solution is not shown to future models — they have to solve the task.
The new item is written to corpus/private/, which is gitignored.
Runs
Past subsample scorecards. Pick models and start a run on the Models tab.
| Run | When | Fingerprint | n | Means |
|---|
Configuration
API keys are not shown. Edit config/local.yaml for a live model.
Checking storage…