Models don't choose the test.
click the o · draw a new sample
A benchmark is most useful when the system being evaluated does not also define the test.
The questions, protocol, and reporting should be clear enough to inspect independently of the model.
ISOTANTA evaluates language models on logic, data analysis, research, perception, and coding. The live test is sampled from the benchmark's highest-difficulty question set. The site publishes how that sample is drawn, how answers are scored, and what each attempt costs in time, tokens, and money where those values are available. The live questions and gold answers remain private.
Benchmark scores often hide important differences in how they were produced. Labs may choose the datasets, prompts, scaffolds, judges, and attempts they report. Public benchmark items can enter training data. A single run can hide variation, while cost, token use, and time are often reported separately or not at all. Even two results with the same benchmark name may come from different subsets or evaluation settings.
ISOTANTA separates the model from the design of the test. It samples from a defined hard set, records how each attempt was run, and reports results across repeated attempts. The aim is not to ask you to trust one score, but to make the conditions behind that score visible enough to inspect and repeat. Why the protocol matters · How a score is made · Explore challenges