Public ranker
Models are ranked by their median score across eligible public attempts. A model is listed only after 5 separate public attempts. The displayed score is the median, not the model’s highest observed score. Each bubble represents one attempt, and the line underneath shows the lowest-to-highest range across attempts. SD and IQR are sit-to-sit spread. Under each row, subject medians (coding, logic, perception, research, data analysis) show where the score comes from. Use the score control to promote a subject into the main number, rank, and sort; attempt bubbles stay overall. Models are tagged by product class: frontier, standard, flash, or open. Cost is billed USD, and token counts are provider-reported. Use search to find a model or lab. Click a row to compare. How this is computed
median
each attempt
range of attempts
frontier closed flagship
standard closed mid
flash speed SKU
open open weights
same score line on every strip
search
score
overall
coding
logic
perception
research
data analysis
sort
score
USD / question
tokens / question
time
USD / 1M tok
name
tier
all
frontier
standard
flash
open
against
none
claude-fable-5
grok-4.6
gpt-5.6-sol
claude-fable-5.1
claude-opus-4.7
gpt-5.6-terra
gpt-6-astra
gpt-5.6-luna-pro
gpt-5.6-sol-pro
grok-4.5
claude-sonnet-4.6
grok-4.7
grok-4.3
claude-opus-4.8
gpt-5.6-terra-pro
gpt-4.1-mini
gpt-5.6-luna
claude-opus-5.5
gpt-5.5
claude-sonnet-5.5
claude-sonnet-5
claude-sonnet-4
gpt-6-astra-pro
o4-mini
kimi-k3
qwen3.7-plus
qwen3.7-max
gpt-5.4
gpt-6.1-sol
gpt-6-luna-pro
ember-1
gpt-6-sol
qwen3-max
glm-5.3-flash
inkling
gpt-6-sol-pro
claude-haiku-5.5
gpt-5.4-nano
mistral-medium-3.1
gpt-6.1-sol-pro
deepseek-v4-pro
command-a
claude-opus-5
grok-4.20
gpt-6-luna
gpt-4.1
mistral-large-4-0
devstral-2512
inkling-small
gpt-5.5-pro
glm-5.3
codestral-2508
llama-4-maverick
mimo-v2.6-pro
deepseek-v4-pro-0813
mistral-medium-3-5
mimo-v2.5-pro
deepseek-v4.1-flash
llama-3.3-70b-instruct
deepseek-r1-0528
gemini-3-flash-preview
mimo-v2.6-flash
deepseek-v4-flash
mimo-v2.5
gemini-2.5-pro
gemini-3.1-flash-lite
deepseek-v4-flash-0731
qwen3.8-2.4t-a95b
muse-spark-1.3
gemini-3.7-flash
gemini-3.8-flash
kimi-k2
kimi-k2.5
gemini-3.5-flash-lite
gemini-2.5-flash
gemini-2.5-flash-lite
gemini-3.5-flash
qwen3.8-flash
gemini-3.6-flash
qwen3.5-flash-02-23
kimi-k2.6
gemini-3.1-pro-preview
mimo-v2.6-pro-ultraspeed
qwen3.6-flash
qwen3.7-flash
qwen3.8-max-0902
muse-glimmer-30b
command-a-plus
qwen3.8-max-prime
glm-5.3-prime
qwen3.8-27b
mistral-small-2603
nova-premier-v1
01
claude-fable-5 frontier
anthropic/claude-fable-5
35 attempts · bubbles above, range below
Mean 97.9
SD 0.89
IQR 0.9
USD / question 0.022
Tokens / question 590
USD / 1M tok 36.39
Time / question 5.7 s
USD / test 3.83
by subject median across 35 sits
coding
97.1
88.8–100.0 · sd 2.3
data analysis
100.0
98.8–100.0 · sd 0.4
logic
96.5
92.9–100.0 · sd 1.7
perception
100.0
100.0–100.0 · sd 0.0
research
98.7
94.7–100.0 · sd 1.4
02
grok-4.6 frontier
x-ai/grok-4.6
58 attempts · bubbles above, range below
Mean 97.3
SD 1.62
IQR 1.3
USD / question 0.0083
Tokens / question 1.6k
USD / 1M tok 5.08
Time / question 11.5 s
USD / test 1.48
by subject median across 58 sits
coding
97.3
79.2–100.0 · sd 3.1
data analysis
100.0
92.0–100.0 · sd 1.3
logic
97.6
91.3–100.0 · sd 1.6
perception
93.2
89.7–100.0 · sd 1.3
research
98.8
91.5–100.0 · sd 1.4
03
gpt-5.6-sol frontier
openai/gpt-5.6-sol
30 attempts · bubbles above, range below
Mean 97.3
SD 1.09
IQR 1.5
USD / question 0.0021
Tokens / question 322
USD / 1M tok 6.58
Time / question 3.5 s
USD / test 0.376
by subject median across 30 sits
coding
97.7
92.1–100.0 · sd 1.9
data analysis
92.7
85.4–96.3 · sd 2.8
logic
97.6
95.3–100.0 · sd 1.3
perception
100.0
98.6–100.0 · sd 0.6
research
100.0
97.3–100.0 · sd 0.8
04
claude-fable-5.1 frontier
anthropic/claude-fable-5.1
10 attempts · bubbles above, range below
Mean 96.8
SD 0.92
IQR 1.6
USD / question 0.019
Tokens / question 553
USD / 1M tok 35.16
Time / question 6.1 s
USD / test 3.44
by subject median across 10 sits
coding
97.7
93.4–100.0 · sd 2.1
data analysis
100.0
98.8–100.0 · sd 0.4
logic
97.6
91.8–100.0 · sd 2.3
perception
87.8
87.8–87.8 · sd 0.0
research
100.0
97.3–100.0 · sd 0.9
05
claude-opus-4.7 frontier
anthropic/claude-opus-4.7
5 attempts · bubbles above, range below
Mean 95.7
SD 1.85
IQR 3.5
USD / question 0.0037
Tokens / question 316
USD / 1M tok 11.93
Time / question 1.9 s
USD / test 0.664
by subject median across 5 sits
coding
97.8
89.7–100.0 · sd 4.6
data analysis
98.8
97.6–100.0 · sd 1.2
logic
97.6
96.5–98.8 · sd 1.2
perception
86.5
86.5–86.5 · sd 0.0
research
98.7
98.7–100.0 · sd 0.7
06
gpt-5.6-terra frontier
openai/gpt-5.6-terra
5 attempts · bubbles above, range below
Mean 96.3
SD 0.36
IQR 0.6
USD / question 0.0029
Tokens / question 360
USD / 1M tok 8.20
Time / question 2.9 s
USD / test 0.522
by subject median across 5 sits
coding
95.2
92.0–95.2 · sd 1.7
data analysis
93.9
91.5–95.1 · sd 1.6
logic
97.6
96.5–100.0 · sd 1.3
perception
100.0
100.0–100.0 · sd 0.0
research
100.0
98.7–100.0 · sd 0.7
07
gpt-6-astra frontier
openai/gpt-6-astra
10 attempts · bubbles above, range below
Mean 96.1
SD 0.93
IQR 1.0
USD / question 0.0083
Tokens / question 276
USD / 1M tok 29.94
Time / question 3.6 s
USD / test 1.46
by subject median across 10 sits
coding
99.7
93.4–100.0 · sd 2.4
data analysis
95.1
91.5–97.6 · sd 2.1
logic
97.6
94.1–100.0 · sd 1.6
perception
86.5
86.5–86.5 · sd 0.0
research
100.0
100.0–100.0 · sd 0.0
08
gpt-5.6-luna-pro frontier
openai/gpt-5.6-luna-pro
5 attempts · bubbles above, range below
Mean 97.0
SD 1.28
IQR 2.2
USD / question 0.0018
Tokens / question 3.9k
USD / 1M tok 0.471
Time / question 6.6 s
USD / test 0.321
by subject median across 5 sits
coding
95.5
95.0–100.0 · sd 2.5
data analysis
92.7
90.2–96.3 · sd 2.3
logic
97.6
95.3–98.8 · sd 1.8
perception
100.0
100.0–100.0 · sd 0.0
research
98.7
97.3–100.0 · sd 1.1
09
gpt-5.6-sol-pro frontier
openai/gpt-5.6-sol-pro
5 attempts · bubbles above, range below
Mean 96.3
SD 1.39
IQR 2.5
USD / question 0.012
Tokens / question 3.3k
USD / 1M tok 3.69
Time / question 6.7 s
USD / test 2.14
by subject median across 5 sits
coding
97.8
95.0–100.0 · sd 2.3
data analysis
91.5
90.2–96.3 · sd 2.4
logic
97.6
95.3–100.0 · sd 1.9
perception
93.2
93.2–93.2 · sd 0.0
research
98.7
97.3–100.0 · sd 0.9
10
grok-4.5 frontier
x-ai/grok-4.5
5 attempts · bubbles above, range below
Mean 95.2
SD 1.28
IQR 2.4
USD / question 0.0065
Tokens / question 1.3k
USD / 1M tok 4.86
Time / question 9.4 s
USD / test 1.14
by subject median across 5 sits
coding
96.6
91.8–100.0 · sd 3.3
data analysis
100.0
98.8–100.0 · sd 0.7
logic
98.8
95.3–100.0 · sd 1.8
perception
81.1
81.1–81.1 · sd 0.0
research
98.7
96.0–98.7 · sd 1.2
11
claude-sonnet-4.6 frontier
anthropic/claude-sonnet-4.6
11 attempts · bubbles above, range below
Mean 93.9
SD 4.56
IQR 3.5
USD / question 0.0026
Tokens / question 287
USD / 1M tok 9.06
Time / question 3.2 s
USD / test 0.285
by subject median across 11 sits
coding
96.4
92.9–100.0 · sd 2.3
data analysis
96.4
89.3–100.0 · sd 3.7
logic
82.1
28.6–100.0 · sd 18.2
perception
100.0
92.9–100.0 · sd 2.5
research
100.0
96.4–100.0 · sd 1.1
12
grok-4.7 frontier
x-ai/grok-4.7
11 attempts · bubbles above, range below
Mean 94.5
SD 0.51
IQR 0.7
USD / question 0.014
Tokens / question 4.1k
USD / 1M tok 3.48
Time / question 10.9 s
USD / test 2.95
by subject median across 11 sits
coding
95.8
91.8–96.2 · sd 1.6
data analysis
100.0
98.9–100.0 · sd 0.5
logic
95.7
94.6–97.8 · sd 1.0
perception
82.1
82.1–82.1 · sd 0.0
research
98.8
97.6–98.8 · sd 0.6
13
grok-4.3 frontier
x-ai/grok-4.3
5 attempts · bubbles above, range below
Mean 95.0
SD 1.41
IQR 2.4
USD / question 0.0017
Tokens / question 934
USD / 1M tok 1.85
Time / question 5.5 s
USD / test 0.306
by subject median across 5 sits
coding
92.3
87.5–97.3 · sd 3.5
data analysis
98.8
98.8–100.0 · sd 0.7
logic
97.6
94.1–100.0 · sd 2.1
perception
87.8
87.8–87.8 · sd 0.0
research
100.0
100.0–100.0 · sd 0.0
14
claude-opus-4.8 frontier
anthropic/claude-opus-4.8
5 attempts · bubbles above, range below
Mean 95.3
SD 1.54
IQR 2.8
USD / question 0.0053
Tokens / question 375
USD / 1M tok 14.27
Time / question 2.8 s
USD / test 0.946
by subject median across 5 sits
coding
97.8
92.4–100.0 · sd 3.2
data analysis
95.1
93.9–98.8 · sd 1.9
logic
98.8
96.5–100.0 · sd 1.5
perception
85.1
83.8–86.5 · sd 1.1
research
98.7
94.7–98.7 · sd 1.7
15
gpt-5.6-terra-pro frontier
openai/gpt-5.6-terra-pro
5 attempts · bubbles above, range below
Mean 94.6
SD 2.02
IQR 3.8
USD / question 0.015
Tokens / question 3.4k
USD / 1M tok 4.29
Time / question 8.0 s
USD / test 2.62
by subject median across 5 sits
coding
92.2
84.8–95.9 · sd 4.6
data analysis
90.2
87.8–96.3 · sd 3.2
logic
96.5
95.3–98.8 · sd 1.6
perception
100.0
100.0–100.0 · sd 0.0
research
98.7
96.0–98.7 · sd 1.2
16
gpt-4.1-mini flash
openai/gpt-4.1-mini
5 attempts · bubbles above, range below
Mean 94.0
SD 0.72
IQR 1.1
USD / question 0.0006
Tokens / question 457
USD / 1M tok 1.22
Time / question 2.6 s
USD / test 0.073
by subject median across 5 sits
coding
91.8
89.3–94.8 · sd 2.2
data analysis
95.2
95.2–95.2 · sd 0.0
logic
96.0
90.0–98.0 · sd 3.3
perception
100.0
100.0–100.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
17
gpt-5.6-luna frontier
openai/gpt-5.6-luna
5 attempts · bubbles above, range below
Mean 94.3
SD 0.53
IQR 1.0
USD / question 0.0003
Tokens / question 401
USD / 1M tok 0.856
Time / question 3.0 s
USD / test 0.061
by subject median across 5 sits
coding
95.2
94.4–97.3 · sd 1.4
data analysis
95.1
93.9–97.6 · sd 1.6
logic
91.8
91.8–92.9 · sd 0.6
perception
86.5
86.5–86.5 · sd 0.0
research
100.0
98.7–100.0 · sd 0.7
18
claude-opus-5.5 frontier
anthropic/claude-opus-5.5
16 attempts · bubbles above, range below
Mean 93.7
SD 1.59
IQR 1.4
USD / question 0.010
Tokens / question 674
USD / 1M tok 15.11
Time / question 6.1 s
USD / test 2.11
by subject median across 16 sits
coding
93.3
88.0–98.5 · sd 2.7
data analysis
100.0
97.7–100.0 · sd 0.7
logic
97.8
90.2–98.9 · sd 2.2
perception
82.1
79.5–82.1 · sd 0.6
research
96.3
87.8–98.8 · sd 2.7
19
gpt-5.5 frontier
openai/gpt-5.5
5 attempts · bubbles above, range below
Mean 94.0
SD 0.75
IQR 1.2
USD / question 0.0098
Tokens / question 443
USD / 1M tok 22.23
Time / question 3.9 s
USD / test 1.74
by subject median across 5 sits
coding
92.0
90.9–95.2 · sd 1.7
data analysis
92.7
91.5–93.9 · sd 1.0
logic
98.8
96.5–100.0 · sd 1.3
perception
87.8
87.8–87.8 · sd 0.0
research
100.0
100.0–100.0 · sd 0.0
20
claude-sonnet-5.5 frontier
anthropic/claude-sonnet-5.5
10 attempts · bubbles above, range below
Mean 93.7
SD 1.84
IQR 3.5
USD / question 0.0047
Tokens / question 642
USD / 1M tok 7.36
Time / question 3.4 s
USD / test 0.625
by subject median across 10 sits
coding
96.8
91.5–100.0 · sd 3.4
data analysis
100.0
100.0–100.0 · sd 0.0
logic
97.0
92.0–100.0 · sd 2.8
perception
69.0
69.0–69.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
21
claude-sonnet-5 frontier
anthropic/claude-sonnet-5
5 attempts · bubbles above, range below
Mean 93.9
SD 0.66
IQR 1.2
USD / question 0.0034
Tokens / question 496
USD / 1M tok 6.78
Time / question 4.6 s
USD / test 0.595
by subject median across 5 sits
coding
94.4
92.0–95.2 · sd 1.5
data analysis
91.5
91.5–92.7 · sd 0.7
logic
97.6
96.5–100.0 · sd 1.3
perception
86.5
86.5–86.5 · sd 0.0
research
98.7
97.3–100.0 · sd 1.1
22
claude-sonnet-4 frontier
anthropic/claude-sonnet-4
5 attempts · bubbles above, range below
Mean 93.4
SD 1.32
IQR 2.2
USD / question 0.0075
Tokens / question 626
USD / 1M tok 12.02
Time / question 5.1 s
USD / test 0.993
by subject median across 5 sits
coding
94.2
89.3–96.7 · sd 2.9
data analysis
100.0
100.0–100.0 · sd 0.0
logic
96.0
94.0–100.0 · sd 2.3
perception
79.3
79.3–82.8 · sd 1.5
research
96.6
96.6–96.6 · sd 0.0
23
gpt-6-astra-pro frontier
openai/gpt-6-astra-pro
5 attempts · bubbles above, range below
Mean 93.8
SD 1.45
IQR 2.6
USD / question 0.036
Tokens / question 2.4k
USD / 1M tok 15.04
Time / question 5.6 s
USD / test 6.40
by subject median across 5 sits
coding
97.8
95.0–100.0 · sd 2.3
data analysis
90.2
86.6–95.1 · sd 3.3
logic
97.6
94.1–100.0 · sd 2.3
perception
81.1
81.1–81.1 · sd 0.0
research
97.3
93.3–98.7 · sd 2.0
24
o4-mini frontier
openai/o4-mini
11 attempts · bubbles above, range below
Mean 91.7
SD 3.99
IQR 1.5
USD / question 0.0026
Tokens / question 694
USD / 1M tok 3.80
Time / question 5.1 s
USD / test 0.288
by subject median across 11 sits
coding
96.4
92.9–100.0 · sd 2.3
data analysis
92.9
85.7–100.0 · sd 5.4
logic
75.0
28.6–100.0 · sd 17.7
perception
96.4
85.7–100.0 · sd 4.5
research
100.0
89.3–100.0 · sd 3.3
25
kimi-k3 frontier
moonshotai/kimi-k3
38 attempts · bubbles above, range below
Mean 92.1
SD 2.55
IQR 2.3
USD / question 0.0088
Tokens / question 819
USD / 1M tok 10.73
Time / question 8.5 s
USD / test 1.56
by subject median across 38 sits
coding
90.8
64.1–94.8 · sd 5.2
data analysis
97.6
86.4–100.0 · sd 2.8
logic
96.5
90.2–100.0 · sd 2.2
perception
81.1
76.9–82.1 · sd 0.7
research
98.7
91.5–100.0 · sd 1.8
26
qwen3.7-plus open
qwen/qwen3.7-plus
5 attempts · bubbles above, range below
Mean 91.4
SD 0.84
IQR 1.5
USD / question 0.0014
Tokens / question 1.2k
USD / 1M tok 1.16
Time / question 13.4 s
USD / test 0.247
by subject median across 5 sits
coding
89.6
87.0–96.5 · sd 3.7
data analysis
91.5
86.6–92.7 · sd 2.4
logic
96.5
96.5–98.8 · sd 1.1
perception
81.1
78.4–81.1 · sd 1.2
research
98.7
94.7–98.7 · sd 1.7
27
qwen3.7-max open
qwen/qwen3.7-max
5 attempts · bubbles above, range below
Mean 90.5
SD 1.67
IQR 3.1
USD / question 0.0049
Tokens / question 1.2k
USD / 1M tok 4.07
Time / question 11.5 s
USD / test 0.873
by subject median across 5 sits
coding
87.4
80.4–88.9 · sd 3.8
data analysis
100.0
97.6–100.0 · sd 1.3
logic
98.8
95.3–98.8 · sd 1.7
perception
75.7
75.7–75.7 · sd 0.0
research
100.0
98.7–100.0 · sd 0.7
28
gpt-5.4 frontier
openai/gpt-5.4
11 attempts · bubbles above, range below
Mean 90.1
SD 4.21
IQR 2.1
USD / question 0.0023
Tokens / question 259
USD / 1M tok 8.66
Time / question 2.1 s
USD / test 0.249
by subject median across 11 sits
coding
100.0
96.4–100.0 · sd 1.8
data analysis
96.4
89.3–100.0 · sd 3.7
logic
64.3
14.3–96.4 · sd 19.6
perception
96.4
85.7–100.0 · sd 4.4
research
96.4
92.9–100.0 · sd 2.3
29
gpt-6.1-sol frontier
openai/gpt-6.1-sol
10 attempts · bubbles above, range below
Mean 88.3
SD 1.67
IQR 3.3
USD / question 0.0022
Tokens / question 333
USD / 1M tok 6.54
Time / question 4.0 s
USD / test 0.288
by subject median across 10 sits
coding
97.2
92.3–100.0 · sd 3.2
data analysis
4.8
4.8–4.8 · sd 0.0
logic
97.0
92.0–100.0 · sd 2.8
perception
86.2
86.2–86.2 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
30
gpt-6-luna-pro frontier
openai/gpt-6-luna-pro
10 attempts · bubbles above, range below
Mean 88.9
SD 1.16
IQR 1.6
USD / question 0.0007
Tokens / question 3.3k
USD / 1M tok 0.203
Time / question 6.6 s
USD / test 0.089
by subject median across 10 sits
coding
94.4
93.8–100.0 · sd 2.1
data analysis
4.8
4.8–4.8 · sd 0.0
logic
94.0
88.0–98.0 · sd 3.2
perception
100.0
100.0–100.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
31
ember-1 standard
fireworks/ember-1
10 attempts · bubbles above, range below
Mean 88.0
SD 1.49
IQR 2.4
USD / question 0.010
Tokens / question 860
USD / 1M tok 11.73
Time / question 7.7 s
USD / test 1.33
by subject median across 10 sits
coding
95.9
92.6–100.0 · sd 2.2
data analysis
42.9
42.9–47.6 · sd 2.5
logic
92.0
84.0–94.0 · sd 3.3
perception
69.0
69.0–69.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
32
gpt-6-sol frontier
openai/gpt-6-sol
10 attempts · bubbles above, range below
Mean 87.9
SD 1.70
IQR 3.4
USD / question 0.0023
Tokens / question 346
USD / 1M tok 6.67
Time / question 3.3 s
USD / test 0.305
by subject median across 10 sits
coding
97.2
92.3–100.0 · sd 3.2
data analysis
4.8
4.8–4.8 · sd 0.0
logic
97.0
92.0–100.0 · sd 2.8
perception
82.8
82.8–82.8 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
33
qwen3-max open
qwen/qwen3-max
11 attempts · bubbles above, range below
Mean 86.4
SD 3.93
IQR 2.9
USD / question 0.0013
Tokens / question 436
USD / 1M tok 2.93
Time / question 5.2 s
USD / test 0.140
by subject median across 11 sits
coding
82.1
78.6–82.1 · sd 1.9
data analysis
96.4
89.3–100.0 · sd 3.7
logic
78.6
28.6–85.7 · sd 16.0
perception
85.7
78.6–85.7 · sd 2.5
research
100.0
100.0–100.0 · sd 0.0
34
glm-5.3-flash flash
z-ai/glm-5.3-flash
30 attempts · bubbles above, range below
Mean 87.7
SD 1.70
IQR 2.1
USD / question 0.0003
Tokens / question 833
USD / 1M tok 0.416
Time / question 6.5 s
USD / test 0.062
by subject median across 30 sits
coding
79.7
70.4–88.2 · sd 4.3
data analysis
95.1
89.0–97.6 · sd 2.4
logic
96.5
91.8–98.8 · sd 1.6
perception
81.1
79.7–81.1 · sd 0.6
research
98.7
97.3–100.0 · sd 0.8
35
inkling standard
thinkingmachines/inkling
30 attempts · bubbles above, range below
Mean 87.8
SD 1.71
IQR 2.4
USD / question 0.0025
Tokens / question 718
USD / 1M tok 3.43
Time / question 7.6 s
USD / test 0.435
by subject median across 30 sits
coding
78.5
69.7–88.9 · sd 4.4
data analysis
92.7
90.2–93.9 · sd 1.0
logic
96.5
94.1–100.0 · sd 1.6
perception
81.1
81.1–81.1 · sd 0.0
research
100.0
97.3–100.0 · sd 0.8
36
gpt-6-sol-pro frontier
openai/gpt-6-sol-pro
10 attempts · bubbles above, range below
Mean 87.8
SD 1.09
IQR 1.8
USD / question 0.0089
Tokens / question 2.6k
USD / 1M tok 3.38
Time / question 5.0 s
USD / test 1.18
by subject median across 10 sits
coding
96.6
94.1–100.0 · sd 1.9
data analysis
4.8
4.8–4.8 · sd 0.0
logic
94.0
88.0–98.0 · sd 3.2
perception
82.8
82.8–82.8 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
37
claude-haiku-5.5 flash
anthropic/claude-haiku-5.5
5 attempts · bubbles above, range below
Mean 86.9
SD 1.18
IQR 2.0
USD / question 0.0003
Tokens / question 859
USD / 1M tok 0.401
Time / question 3.1 s
USD / test 0.045
by subject median across 5 sits
coding
87.4
84.6–90.5 · sd 2.5
data analysis
100.0
100.0–100.0 · sd 0.0
logic
86.0
84.0–88.0 · sd 1.7
perception
69.0
69.0–69.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
38
gpt-5.4-nano flash
openai/gpt-5.4-nano
5 attempts · bubbles above, range below
Mean 86.1
SD 1.43
IQR 2.5
USD / question 0.0004
Tokens / question 408
USD / 1M tok 0.879
Time / question 2.0 s
USD / test 0.047
by subject median across 5 sits
coding
91.1
87.2–97.0 · sd 3.6
data analysis
61.9
61.9–66.7 · sd 2.6
logic
84.0
76.0–92.0 · sd 6.1
perception
75.9
75.9–86.2 · sd 4.5
research
89.7
75.9–96.6 · sd 7.9
39
mistral-medium-3.1 open
mistralai/mistral-medium-3.1
30 attempts · bubbles above, range below
Mean 86.6
SD 1.19
IQR 2.1
USD / question 0.0007
Tokens / question 472
USD / 1M tok 1.49
Time / question 1.6 s
USD / test 0.124
by subject median across 30 sits
coding
94.2
87.6–98.8 · sd 3.4
data analysis
89.0
84.1–91.5 · sd 1.8
logic
50.6
48.2–54.1 · sd 1.4
perception
100.0
100.0–100.0 · sd 0.0
research
94.7
90.7–94.7 · sd 0.9
40
gpt-6.1-sol-pro frontier
openai/gpt-6.1-sol-pro
10 attempts · bubbles above, range below
Mean 86.6
SD 1.11
IQR 1.8
USD / question 0.0082
Tokens / question 2.6k
USD / 1M tok 3.19
Time / question 6.7 s
USD / test 1.09
by subject median across 10 sits
coding
96.6
94.1–100.0 · sd 1.9
data analysis
4.8
4.8–4.8 · sd 0.0
logic
96.0
90.0–100.0 · sd 3.2
perception
69.0
69.0–69.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
41
deepseek-v4-pro open
deepseek/deepseek-v4-pro
5 attempts · bubbles above, range below
Mean 85.5
SD 2.76
IQR 5.3
USD / question 0.0018
Tokens / question 828
USD / 1M tok 2.14
Time / question 9.8 s
USD / test 0.315
by subject median across 5 sits
coding
71.5
62.5–77.8 · sd 6.1
data analysis
98.8
97.6–100.0 · sd 1.0
logic
94.1
90.6–97.6 · sd 2.8
perception
81.1
79.7–81.1 · sd 0.7
research
100.0
100.0–100.0 · sd 0.0
42
command-a open
cohere/command-a
30 attempts · bubbles above, range below
Mean 84.5
SD 1.80
IQR 2.6
USD / question 0.0028
Tokens / question 376
USD / 1M tok 7.32
Time / question 4.4 s
USD / test 0.487
by subject median across 30 sits
coding
78.1
67.7–86.4 · sd 4.7
data analysis
96.3
93.9–100.0 · sd 2.0
logic
72.9
68.2–76.5 · sd 2.1
perception
87.8
86.5–87.8 · sd 0.5
research
99.3
96.0–100.0 · sd 1.1
43
claude-opus-5 frontier
anthropic/claude-opus-5
15 attempts · bubbles above, range below
Mean 85.2
SD 1.55
IQR 2.3
USD / question 0.017
Tokens / question 830
USD / 1M tok 20.11
Time / question 6.1 s
USD / test 2.96
by subject median across 15 sits
coding
64.0
57.7–72.2 · sd 4.0
data analysis
100.0
98.8–100.0 · sd 0.4
logic
98.8
96.5–100.0 · sd 1.1
perception
93.2
91.9–93.2 · sd 0.6
research
100.0
96.0–100.0 · sd 1.1
44
grok-4.20 frontier
x-ai/grok-4.20
10 attempts · bubbles above, range below
Mean 85.0
SD 2.99
IQR 5.8
USD / question 0.0009
Tokens / question 581
USD / 1M tok 1.52
Time / question 2.9 s
USD / test 0.116
by subject median across 10 sits
coding
87.0
81.2–95.8 · sd 4.6
data analysis
100.0
100.0–100.0 · sd 0.0
logic
77.0
72.0–84.0 · sd 4.0
perception
65.5
58.6–69.0 · sd 3.6
research
96.6
93.1–96.6 · sd 1.1
45
gpt-6-luna frontier
openai/gpt-6-luna
10 attempts · bubbles above, range below
Mean 85.2
SD 1.47
IQR 2.0
USD / question 0.0002
Tokens / question 485
USD / 1M tok 0.381
Time / question 3.5 s
USD / test 0.024
by subject median across 10 sits
coding
90.1
87.4–96.5 · sd 2.7
data analysis
4.8
4.8–4.8 · sd 0.0
logic
88.0
84.0–92.0 · sd 2.9
perception
100.0
100.0–100.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
46
gpt-4.1 standard
openai/gpt-4.1
5 attempts · bubbles above, range below
Mean 85.0
SD 1.27
IQR 2.0
USD / question 0.0028
Tokens / question 455
USD / 1M tok 6.07
Time / question 2.6 s
USD / test 0.364
by subject median across 5 sits
coding
88.6
87.6–94.8 · sd 2.9
data analysis
85.7
85.7–85.7 · sd 0.0
logic
66.0
64.0–66.0 · sd 1.1
perception
82.8
82.8–82.8 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
47
mistral-large-4-0 open
mistralai/mistral-large-4-0
10 attempts · bubbles above, range below
Mean 84.3
SD 2.39
IQR 2.3
USD / question 0.0008
Tokens / question 492
USD / 1M tok 1.65
Time / question 3.6 s
USD / test 0.107
by subject median across 10 sits
coding
84.7
76.8–93.0 · sd 4.5
data analysis
64.3
61.9–66.7 · sd 2.5
logic
95.0
86.0–98.0 · sd 3.8
perception
69.0
69.0–69.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
48
devstral-2512 standard
mistralai/devstral-2512
30 attempts · bubbles above, range below
Mean 83.4
SD 1.59
IQR 2.6
USD / question 0.0007
Tokens / question 475
USD / 1M tok 1.49
Time / question 1.5 s
USD / test 0.125
by subject median across 30 sits
coding
93.0
84.7–98.8 · sd 4.2
data analysis
89.0
84.1–91.5 · sd 1.9
logic
47.1
43.5–49.4 · sd 1.3
perception
86.5
86.5–86.5 · sd 0.0
research
94.7
92.0–94.7 · sd 0.7
49
inkling-small flash
thinkingmachines/inkling-small
30 attempts · bubbles above, range below
Mean 82.3
SD 2.01
IQR 3.3
USD / question 0.0008
Tokens / question 756
USD / 1M tok 1.06
Time / question 2.4 s
USD / test 0.142
by subject median across 30 sits
coding
62.9
54.3–73.0 · sd 5.3
data analysis
93.9
89.0–96.3 · sd 2.0
logic
96.5
92.9–98.8 · sd 1.8
perception
81.1
81.1–81.1 · sd 0.0
research
98.7
98.7–100.0 · sd 0.7
50
gpt-5.5-pro frontier
openai/gpt-5.5-pro
5 attempts · bubbles above, range below
Mean 81.4
SD 1.73
IQR 3.1
USD / question 0.104
Tokens / question 690
USD / 1M tok 150.56
Time / question 13.1 s
USD / test 18.36
by subject median across 5 sits
coding
63.7
58.7–66.5 · sd 2.9
data analysis
90.2
85.4–95.1 · sd 4.4
logic
95.3
94.1–100.0 · sd 2.3
perception
81.1
81.1–81.1 · sd 0.0
research
100.0
98.7–100.0 · sd 0.7
51
glm-5.3 standard
z-ai/glm-5.3
33 attempts · bubbles above, range below
Mean 80.6
SD 1.99
IQR 1.7
USD / question 0.0032
Tokens / question 894
USD / 1M tok 3.57
Time / question 4.9 s
USD / test 0.570
by subject median across 33 sits
coding
64.0
51.6–71.4 · sd 4.3
data analysis
92.7
86.4–98.9 · sd 2.0
logic
94.1
84.8–97.6 · sd 2.9
perception
81.1
76.9–88.5 · sd 1.6
research
92.0
89.3–95.1 · sd 1.3
52
codestral-2508 standard
mistralai/codestral-2508
30 attempts · bubbles above, range below
Mean 79.5
SD 1.71
IQR 2.5
USD / question 0.0003
Tokens / question 377
USD / 1M tok 0.679
Time / question 1.8 s
USD / test 0.045
by subject median across 30 sits
coding
89.3
81.1–95.2 · sd 4.0
data analysis
51.8
47.6–56.1 · sd 2.3
logic
80.0
76.5–85.9 · sd 2.5
perception
83.8
79.7–87.8 · sd 2.0
research
82.7
77.3–88.0 · sd 2.9
53
llama-4-maverick open
meta-llama/llama-4-maverick
30 attempts · bubbles above, range below
Mean 79.3
SD 1.34
IQR 1.9
USD / question 0.0004
Tokens / question 599
USD / 1M tok 0.637
Time / question 6.1 s
USD / test 0.068
by subject median across 30 sits
coding
93.7
86.6–98.9 · sd 3.8
data analysis
70.7
64.6–74.4 · sd 2.3
logic
92.9
84.7–97.6 · sd 2.9
perception
58.8
52.7–62.2 · sd 2.5
research
62.7
58.7–66.7 · sd 2.0
54
mimo-v2.6-pro standard
xiaomi/mimo-v2.6-pro
10 attempts · bubbles above, range below
Mean 78.7
SD 0.97
IQR 1.8
USD / question 0.0007
Tokens / question 926
USD / 1M tok 0.803
Time / question 18.5 s
USD / test 0.155
by subject median across 10 sits
coding
66.7
62.9–69.4 · sd 2.5
data analysis
86.9
85.2–90.9 · sd 1.9
logic
91.8
90.2–94.6 · sd 1.6
perception
76.3
71.8–76.9 · sd 1.9
research
85.4
84.1–86.6 · sd 1.0
55
deepseek-v4-pro-0813 open
deepseek/deepseek-v4-pro-0813
8 attempts · bubbles above, range below
Mean 76.7
SD 3.79
IQR 6.2
USD / question 0.0017
Tokens / question 763
USD / 1M tok 2.37
Time / question 6.8 s
USD / test 0.313
by subject median across 8 sits
coding
54.3
47.4–65.9 · sd 5.7
data analysis
97.0
84.1–100.0 · sd 5.8
logic
85.9
68.5–89.4 · sd 8.2
perception
81.1
79.7–88.5 · sd 3.2
research
99.4
92.7–100.0 · sd 2.6
56
mistral-medium-3-5 open
mistralai/mistral-medium-3-5
10 attempts · bubbles above, range below
Mean 78.7
SD 2.90
IQR 4.9
USD / question 0.0033
Tokens / question 568
USD / 1M tok 5.83
Time / question 1.7 s
USD / test 0.437
by subject median across 10 sits
coding
79.5
75.6–89.8 · sd 5.1
data analysis
38.1
38.1–42.9 · sd 2.5
logic
65.0
60.0–68.0 · sd 2.7
perception
100.0
100.0–100.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
57
mimo-v2.5-pro standard
xiaomi/mimo-v2.5-pro
5 attempts · bubbles above, range below
Mean 77.0
SD 2.13
IQR 3.7
USD / question 0.0008
Tokens / question 1.1k
USD / 1M tok 0.752
Time / question 17.7 s
USD / test 0.107
by subject median across 5 sits
coding
67.1
61.1–73.0 · sd 4.7
data analysis
90.5
90.5–90.5 · sd 0.0
logic
94.0
90.0–98.0 · sd 3.4
perception
65.5
58.6–69.0 · sd 4.2
research
96.6
93.1–96.6 · sd 1.5
58
deepseek-v4.1-flash flash
deepseek/deepseek-v4.1-flash
5 attempts · bubbles above, range below
Mean 76.6
SD 1.06
IQR 2.0
USD / question 0.0006
Tokens / question 748
USD / 1M tok 0.801
Time / question 4.7 s
USD / test 0.106
by subject median across 5 sits
coding
47.0
45.7–49.2 · sd 1.6
data analysis
95.1
93.9–98.8 · sd 2.0
logic
96.5
91.8–96.5 · sd 2.3
perception
81.1
81.1–81.1 · sd 0.0
research
100.0
100.0–100.0 · sd 0.0
59
llama-3.3-70b-instruct open
meta-llama/llama-3.3-70b-instruct
5 attempts · bubbles above, range below
Mean 74.8
SD 3.10
IQR 5.6
USD / question 0.0002
Tokens / question 526
USD / 1M tok 0.421
Time / question 10.7 s
USD / test 0.029
by subject median across 5 sits
coding
73.7
62.3–74.8 · sd 5.6
data analysis
85.7
85.7–85.7 · sd 0.0
logic
56.0
54.0–60.0 · sd 2.6
perception
100.0
100.0–100.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
60
deepseek-r1-0528 open
deepseek/deepseek-r1-0528
11 attempts · bubbles above, range below
Mean 72.5
SD 5.43
IQR 2.9
USD / question 0.0044
Tokens / question 2.1k
USD / 1M tok 2.10
Time / question 62.1 s
USD / test 0.489
by subject median across 11 sits
coding
39.3
35.7–50.0 · sd 4.0
data analysis
60.7
42.9–67.9 · sd 6.7
logic
78.6
28.6–96.4 · sd 18.0
perception
85.7
78.6–100.0 · sd 5.5
research
100.0
96.4–100.0 · sd 1.4
61
gemini-3-flash-preview flash
google/gemini-3-flash-preview
30 attempts · bubbles above, range below
Mean 76.6
SD 9.57
IQR 4.0
USD / question 0.0006
Tokens / question 286
USD / 1M tok 2.21
Time / question 3.6 s
USD / test 0.064
by subject median across 30 sits
coding
100.0
100.0–100.0 · sd 0.0
data analysis
69.6
53.6–100.0 · sd 13.4
logic
71.4
57.1–100.0 · sd 11.6
perception
71.4
57.1–100.0 · sd 12.0
research
67.9
46.4–100.0 · sd 14.6
62
mimo-v2.6-flash flash
xiaomi/mimo-v2.6-flash
10 attempts · bubbles above, range below
Mean 73.2
SD 2.21
IQR 3.0
USD / question 0.0002
Tokens / question 860
USD / 1M tok 0.262
Time / question 13.7 s
USD / test 0.030
by subject median across 10 sits
coding
72.1
63.1–76.5 · sd 3.8
data analysis
23.8
19.0–23.8 · sd 2.0
logic
93.0
86.0–96.0 · sd 3.4
perception
65.5
58.6–69.0 · sd 3.2
research
93.1
89.7–96.6 · sd 2.8
63
deepseek-v4-flash flash
deepseek/deepseek-v4-flash
30 attempts · bubbles above, range below
Mean 75.3
SD 9.15
IQR 6.1
USD / question 0.0001
Tokens / question 391
USD / 1M tok 0.230
Time / question 7.6 s
USD / test 0.0091
by subject median across 30 sits
coding
94.7
94.7–94.7 · sd 0.0
data analysis
75.0
53.6–100.0 · sd 12.6
logic
69.6
53.6–100.0 · sd 12.0
perception
60.7
42.9–85.7 · sd 11.8
research
75.0
53.6–100.0 · sd 13.2
64
mimo-v2.5 standard
xiaomi/mimo-v2.5
5 attempts · bubbles above, range below
Mean 73.1
SD 2.81
IQR 4.8
USD / question 0.0002
Tokens / question 822
USD / 1M tok 0.300
Time / question 11.8 s
USD / test 0.032
by subject median across 5 sits
coding
71.1
70.5–81.6 · sd 4.8
data analysis
14.3
9.5–14.3 · sd 2.1
logic
90.0
88.0–92.0 · sd 1.7
perception
65.5
62.1–65.5 · sd 1.5
research
89.7
86.2–93.1 · sd 2.9
65
gemini-2.5-pro frontier
google/gemini-2.5-pro
11 attempts · bubbles above, range below
Mean 69.8
SD 3.84
IQR 1.4
USD / question 0.022
Tokens / question 2.3k
USD / 1M tok 9.50
Time / question 20.7 s
USD / test 2.43
by subject median across 11 sits
coding
0.0
0.0–3.6 · sd 1.1
data analysis
92.9
89.3–100.0 · sd 3.9
logic
75.0
28.6–89.3 · sd 16.3
perception
82.1
78.6–85.7 · sd 3.1
research
100.0
92.9–100.0 · sd 2.1
66
gemini-3.1-flash-lite flash
google/gemini-3.1-flash-lite
30 attempts · bubbles above, range below
Mean 73.9
SD 9.27
IQR 4.0
USD / question 0.0003
Tokens / question 285
USD / 1M tok 1.09
Time / question 2.0 s
USD / test 0.031
by subject median across 30 sits
coding
73.7
73.7–73.7 · sd 0.0
data analysis
67.9
53.6–100.0 · sd 13.1
logic
73.2
57.1–100.0 · sd 11.8
perception
75.0
57.1–100.0 · sd 11.8
research
67.9
50.0–100.0 · sd 13.5
67
deepseek-v4-flash-0731 flash
deepseek/deepseek-v4-flash-0731
30 attempts · bubbles above, range below
Mean 72.7
SD 8.35
IQR 5.9
USD / question 0.0001
Tokens / question 699
USD / 1M tok 0.186
Time / question 5.0 s
USD / test 0.015
by subject median across 30 sits
coding
68.4
68.4–68.4 · sd 0.0
data analysis
75.0
53.6–100.0 · sd 12.1
logic
69.6
50.0–96.4 · sd 12.4
perception
60.7
46.4–85.7 · sd 10.4
research
82.1
60.7–100.0 · sd 11.7
68
qwen3.8-2.4t-a95b open
qwen/qwen3.8-2.4t-a95b
5 attempts · bubbles above, range below
Mean 70.9
SD 1.81
IQR 2.6
USD / question 0.0052
Tokens / question 989
USD / 1M tok 5.23
Time / question 7.2 s
USD / test 0.917
by subject median across 5 sits
coding
30.9
29.3–39.8 · sd 4.1
data analysis
96.3
91.5–100.0 · sd 3.3
logic
95.3
92.9–96.5 · sd 1.6
perception
81.1
81.1–81.1 · sd 0.0
research
97.3
94.7–100.0 · sd 1.9
69
muse-spark-1.3 standard
meta/muse-spark-1.3
6 attempts · bubbles above, range below
Mean 59.2
SD 29.06
IQR 20.5
USD / question 0.0053
Tokens / question 1.3k
USD / 1M tok 3.91
Time / question 10.1 s
USD / test 0.933
by subject median across 6 sits
coding
31.1
0.0–40.4 · sd 14.0
data analysis
94.5
0.0–97.6 · sd 38.9
logic
96.5
0.0–98.8 · sd 39.6
perception
80.4
0.0–81.1 · sd 32.9
research
97.3
0.0–98.7 · sd 39.8
70
gemini-3.7-flash flash
google/gemini-3.7-flash
30 attempts · bubbles above, range below
Mean 72.5
SD 9.75
IQR 5.2
USD / question 0.0014
Tokens / question 445
USD / 1M tok 3.14
Time / question 5.5 s
USD / test 0.142
by subject median across 30 sits
coding
100.0
100.0–100.0 · sd 0.0
data analysis
69.6
53.6–100.0 · sd 13.4
logic
67.9
50.0–100.0 · sd 12.5
perception
55.4
42.9–85.7 · sd 12.3
research
67.9
46.4–100.0 · sd 14.6
71
gemini-3.8-flash flash
google/gemini-3.8-flash
31 attempts · bubbles above, range below
Mean 72.8
SD 10.65
IQR 6.1
USD / question 0.0018
Tokens / question 536
USD / 1M tok 3.24
Time / question 5.4 s
USD / test 0.177
by subject median across 31 sits
coding
73.7
73.7–100.0 · sd 4.7
data analysis
71.4
53.6–100.0 · sd 14.1
logic
71.4
53.6–100.0 · sd 12.5
perception
71.4
57.1–100.0 · sd 13.1
research
67.9
46.4–100.0 · sd 15.4
72
kimi-k2 standard
moonshotai/kimi-k2
30 attempts · bubbles above, range below
Mean 72.0
SD 9.53
IQR 4.0
USD / question 0.0012
Tokens / question 598
USD / 1M tok 2.02
Time / question 13.5 s
USD / test 0.122
by subject median across 30 sits
coding
94.7
94.7–94.7 · sd 0.0
data analysis
67.9
46.4–100.0 · sd 13.6
logic
69.6
53.6–100.0 · sd 12.3
perception
71.4
57.1–100.0 · sd 11.8
research
53.6
32.1–85.7 · sd 14.4
73
kimi-k2.5 standard
moonshotai/kimi-k2.5
30 attempts · bubbles above, range below
Mean 72.0
SD 9.20
IQR 5.8
USD / question 0.0046
Tokens / question 2.1k
USD / 1M tok 2.23
Time / question 62.4 s
USD / test 0.464
by subject median across 30 sits
coding
78.9
78.9–78.9 · sd 0.0
data analysis
64.3
46.4–92.9 · sd 12.7
logic
71.4
50.0–100.0 · sd 11.7
perception
71.4
57.1–100.0 · sd 11.8
research
67.9
46.4–100.0 · sd 14.4
74
gemini-3.5-flash-lite flash
google/gemini-3.5-flash-lite
30 attempts · bubbles above, range below
Mean 69.9
SD 9.41
IQR 4.8
USD / question 0.0006
Tokens / question 315
USD / 1M tok 1.86
Time / question 1.8 s
USD / test 0.060
by subject median across 30 sits
coding
78.9
78.9–78.9 · sd 0.0
data analysis
69.6
53.6–100.0 · sd 13.4
logic
69.6
50.0–92.9 · sd 11.8
perception
57.1
42.9–85.7 · sd 11.9
research
67.9
46.4–100.0 · sd 14.3
75
gemini-2.5-flash flash
google/gemini-2.5-flash
30 attempts · bubbles above, range below
Mean 68.7
SD 9.70
IQR 5.5
USD / question 0.0009
Tokens / question 449
USD / 1M tok 2.05
Time / question 4.7 s
USD / test 0.093
by subject median across 30 sits
coding
68.4
68.4–68.4 · sd 0.0
data analysis
69.6
53.6–100.0 · sd 13.4
logic
71.4
50.0–100.0 · sd 12.4
perception
57.1
42.9–85.7 · sd 12.0
research
67.9
46.4–100.0 · sd 14.6
76
gemini-2.5-flash-lite flash
google/gemini-2.5-flash-lite
30 attempts · bubbles above, range below
Mean 68.6
SD 9.03
IQR 6.3
USD / question 0.0002
Tokens / question 502
USD / 1M tok 0.342
Time / question 1.9 s
USD / test 0.017
by subject median across 30 sits
coding
78.9
78.9–78.9 · sd 0.0
data analysis
71.4
53.6–100.0 · sd 12.3
logic
67.9
42.9–96.4 · sd 11.4
perception
60.7
42.9–85.7 · sd 11.8
research
60.7
39.3–85.7 · sd 13.2
77
gemini-3.5-flash flash
google/gemini-3.5-flash
30 attempts · bubbles above, range below
Mean 68.3
SD 9.74
IQR 4.4
USD / question 0.0066
Tokens / question 806
USD / 1M tok 8.16
Time / question 8.2 s
USD / test 0.662
by subject median across 30 sits
coding
73.7
73.7–73.7 · sd 0.0
data analysis
69.6
53.6–100.0 · sd 13.4
logic
69.6
50.0–100.0 · sd 12.1
perception
53.6
42.9–85.7 · sd 12.5
research
67.9
46.4–100.0 · sd 14.6
78
qwen3.8-flash flash
qwen/qwen3.8-flash
30 attempts · bubbles above, range below
Mean 68.4
SD 9.09
IQR 4.5
USD / question 0.0003
Tokens / question 683
USD / 1M tok 0.407
Time / question 10.6 s
USD / test 0.028
by subject median across 30 sits
coding
63.2
63.2–63.2 · sd 0.0
data analysis
71.4
53.6–100.0 · sd 12.6
logic
67.9
50.0–92.9 · sd 11.6
perception
57.1
42.9–85.7 · sd 11.5
research
71.4
53.6–100.0 · sd 13.1
79
gemini-3.6-flash flash
google/gemini-3.6-flash
30 attempts · bubbles above, range below
Mean 67.2
SD 9.53
IQR 5.4
USD / question 0.0027
Tokens / question 788
USD / 1M tok 3.40
Time / question 8.1 s
USD / test 0.272
by subject median across 30 sits
coding
68.4
68.4–68.4 · sd 0.0
data analysis
69.6
53.6–100.0 · sd 13.1
logic
69.6
50.0–100.0 · sd 12.3
perception
55.4
42.9–85.7 · sd 12.5
research
64.3
46.4–96.4 · sd 13.4
80
qwen3.5-flash-02-23 flash
qwen/qwen3.5-flash-02-23
30 attempts · bubbles above, range below
Mean 66.7
SD 8.80
IQR 5.0
USD / question 0.0004
Tokens / question 1.8k
USD / 1M tok 0.248
Time / question 19.9 s
USD / test 0.044
by subject median across 30 sits
coding
100.0
100.0–100.0 · sd 0.0
data analysis
64.3
42.9–96.4 · sd 11.8
logic
39.3
21.4–67.9 · sd 11.7
perception
58.9
42.9–85.7 · sd 11.9
research
75.0
53.6–100.0 · sd 13.2
81
kimi-k2.6 standard
moonshotai/kimi-k2.6
30 attempts · bubbles above, range below
Mean 64.0
SD 9.03
IQR 5.9
USD / question 0.0036
Tokens / question 1.2k
USD / 1M tok 3.00
Time / question 36.8 s
USD / test 0.366
by subject median across 30 sits
coding
78.9
78.9–78.9 · sd 0.0
data analysis
50.0
28.6–82.1 · sd 12.5
logic
64.3
46.4–100.0 · sd 11.5
perception
57.1
42.9–85.7 · sd 11.9
research
67.9
42.9–100.0 · sd 13.9
82
gemini-3.1-pro-preview frontier
google/gemini-3.1-pro-preview
35 attempts · bubbles above, range below
Mean 58.2
SD 1.31
IQR 1.9
USD / question 0.013
Tokens / question 1.2k
USD / 1M tok 10.92
Time / question 9.3 s
USD / test 2.39
by subject median across 35 sits
coding
6.4
2.6–9.7 · sd 2.0
data analysis
85.4
80.5–87.8 · sd 1.7
logic
87.1
83.5–90.6 · sd 1.8
perception
81.1
81.1–81.1 · sd 0.0
research
100.0
98.7–100.0 · sd 0.6
83
mimo-v2.6-pro-ultraspeed standard
xiaomi/mimo-v2.6-pro-ultraspeed
10 attempts · bubbles above, range below
Mean 55.6
SD 2.76
IQR 3.9
USD / question 0.0099
Tokens / question 1.2k
USD / 1M tok 8.15
Time / question 4.2 s
USD / test 1.31
by subject median across 10 sits
coding
40.8
30.2–44.3 · sd 5.2
data analysis
35.7
33.3–38.1 · sd 2.5
logic
93.0
86.0–98.0 · sd 3.3
perception
55.2
44.8–69.0 · sd 7.3
research
87.9
86.2–96.6 · sd 4.1
84
qwen3.6-flash flash
qwen/qwen3.6-flash
30 attempts · bubbles above, range below
Mean 58.9
SD 9.12
IQR 6.8
USD / question 0.0016
Tokens / question 1.5k
USD / 1M tok 1.06
Time / question 15.1 s
USD / test 0.163
by subject median across 30 sits
coding
73.7
73.7–73.7 · sd 0.0
data analysis
50.0
32.1–78.6 · sd 12.7
logic
50.0
35.7–78.6 · sd 11.5
perception
58.9
42.9–85.7 · sd 11.9
research
58.9
35.7–85.7 · sd 13.6
85
qwen3.7-flash flash
qwen/qwen3.7-flash
30 attempts · bubbles above, range below
Mean 56.9
SD 8.30
IQR 5.6
USD / question 0.0002
Tokens / question 1.7k
USD / 1M tok 0.123
Time / question 23.4 s
USD / test 0.021
by subject median across 30 sits
coding
78.9
78.9–78.9 · sd 0.0
data analysis
37.5
21.4–67.9 · sd 11.6
logic
48.2
35.7–75.0 · sd 10.7
perception
60.7
42.9–85.7 · sd 11.8
research
57.1
35.7–85.7 · sd 12.8
86
qwen3.8-max-0902 open
qwen/qwen3.8-max-0902
13 attempts · bubbles above, range below
Mean 56.3
SD 7.06
IQR 9.9
USD / question 0.0056
Tokens / question 1.1k
USD / 1M tok 5.28
Time / question 10.6 s
USD / test 0.746
by subject median across 13 sits
coding
25.0
16.7–35.7 · sd 5.2
data analysis
90.5
89.8–97.7 · sd 2.7
logic
90.0
84.0–96.0 · sd 3.5
perception
69.0
69.0–82.1 · sd 5.1
research
96.6
86.6–96.6 · sd 2.8
87
muse-glimmer-30b standard
meta/muse-glimmer-30b
10 attempts · bubbles above, range below
Mean 52.0
SD 2.20
IQR 4.1
USD / question 0.0014
Tokens / question 1.2k
USD / 1M tok 1.15
Time / question 16.3 s
USD / test 0.182
by subject median across 10 sits
coding
32.8
26.4–38.3 · sd 3.4
data analysis
14.3
9.5–14.3 · sd 2.0
logic
90.0
84.0–92.0 · sd 3.0
perception
69.0
65.5–69.0 · sd 1.7
research
96.6
96.6–96.6 · sd 0.0
88
command-a-plus open
cohere/command-a-plus
10 attempts · bubbles above, range below
Mean 51.1
SD 2.28
IQR 3.5
USD / question 0.0020
Tokens / question 1.5k
USD / 1M tok 1.39
Time / question 10.2 s
USD / test 0.267
by subject median across 10 sits
coding
33.1
28.1–37.5 · sd 3.1
data analysis
42.9
42.9–47.6 · sd 2.3
logic
71.0
64.0–76.0 · sd 4.0
perception
69.0
65.5–69.0 · sd 1.4
research
93.1
89.7–96.6 · sd 2.3
89
qwen3.8-max-prime open
qwen/qwen3.8-max-prime
10 attempts · bubbles above, range below
Mean 50.7
SD 1.88
IQR 3.4
USD / question 0.012
Tokens / question 1.1k
USD / 1M tok 10.60
Time / question 6.0 s
USD / test 1.54
by subject median across 10 sits
coding
17.1
13.3–22.4 · sd 3.2
data analysis
95.2
95.2–100.0 · sd 1.5
logic
89.0
82.0–94.0 · sd 3.5
perception
69.0
69.0–69.0 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
90
glm-5.3-prime standard
z-ai/glm-5.3-prime
10 attempts · bubbles above, range below
Mean 47.2
SD 2.44
IQR 3.2
USD / question 0.0094
Tokens / question 1.2k
USD / 1M tok 8.04
Time / question 8.4 s
USD / test 1.24
by subject median across 10 sits
coding
10.0
7.0–17.2 · sd 3.0
data analysis
100.0
100.0–100.0 · sd 0.0
logic
88.0
86.0–94.0 · sd 2.6
perception
67.2
65.5–69.0 · sd 1.8
research
96.6
93.1–96.6 · sd 1.7
91
qwen3.8-27b open
qwen/qwen3.8-27b
5 attempts · bubbles above, range below
Mean 46.2
SD 2.18
IQR 4.1
USD / question 0.0024
Tokens / question 1.2k
USD / 1M tok 2.01
Time / question 10.1 s
USD / test 0.322
by subject median across 5 sits
coding
6.8
6.0–10.9 · sd 2.1
data analysis
85.7
85.7–90.5 · sd 2.1
logic
90.0
84.0–92.0 · sd 3.3
perception
82.8
82.8–82.8 · sd 0.0
research
96.6
96.6–96.6 · sd 0.0
92
mistral-small-2603 flash
mistralai/mistral-small-2603
30 attempts · bubbles above, range below
Mean 27.3
SD 0.99
IQR 1.5
USD / question 0.0001
Tokens / question 229
USD / 1M tok 0.450
Time / question 39.3 s
USD / test 0.018
by subject median across 30 sits
coding
16.9
12.7–22.9 · sd 2.2
data analysis
41.5
36.6–45.1 · sd 2.4
logic
25.9
21.2–31.8 · sd 2.5
perception
31.1
23.0–35.1 · sd 2.6
research
34.7
28.0–40.0 · sd 2.9
93
nova-premier-v1 frontier
amazon/nova-premier-v1
30 attempts · bubbles above, range below
Mean 2.5
SD 2.30
IQR 1.2
USD / question 0.0002
Tokens / question 19
USD / 1M tok 9.37
Time / question 39.3 s
USD / test 0.033
by subject median across 30 sits
coding
4.9
2.2–29.9 · sd 4.9
data analysis
2.4
0.0–14.6 · sd 2.6
logic
0.0
0.0–3.5 · sd 1.0
perception
0.0
0.0–0.0 · sd 0.0
research
0.0
0.0–0.0 · sd 0.0