ISOTANTA
ink
field

Same test. Same rules.

public ranker

Public ranker

Models are ranked by their median score across eligible public attempts. A model is listed only after 5 separate public attempts. The displayed score is the median, not the model’s highest observed score. Each bubble represents one attempt, and the line underneath shows the lowest-to-highest range across attempts. SD and IQR are sit-to-sit spread. Under each row, subject medians (coding, logic, perception, research, data analysis) show where the score comes from. Use the score control to promote a subject into the main number, rank, and sort; attempt bubbles stay overall. Models are tagged by product class: frontier, standard, flash, or open. Cost is billed USD, and token counts are provider-reported. Use search to find a model or lab. Click a row to compare. How this is computed

median each attempt range of attempts frontier closed flagship standard closed mid flash speed SKU open open weights same score line on every strip

01

claude-fable-5 frontier

anthropic/claude-fable-5

98.0

median score

35 attempts · bubbles above, range below

Mean
97.9
SD
0.89
IQR
0.9
USD / question
0.022
Tokens / question
590
USD / 1M tok
36.39
Time / question
5.7 s
USD / test
3.83

by subject median across 35 sits

  • coding 97.1 88.8–100.0 · sd 2.3
  • data analysis 100.0 98.8–100.0 · sd 0.4
  • logic 96.5 92.9–100.0 · sd 1.7
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 98.7 94.7–100.0 · sd 1.4

02

grok-4.6 frontier

x-ai/grok-4.6

97.6

median score

58 attempts · bubbles above, range below

Mean
97.3
SD
1.62
IQR
1.3
USD / question
0.0083
Tokens / question
1.6k
USD / 1M tok
5.08
Time / question
11.5 s
USD / test
1.48

by subject median across 58 sits

  • coding 97.3 79.2–100.0 · sd 3.1
  • data analysis 100.0 92.0–100.0 · sd 1.3
  • logic 97.6 91.3–100.0 · sd 1.6
  • perception 93.2 89.7–100.0 · sd 1.3
  • research 98.8 91.5–100.0 · sd 1.4

03

gpt-5.6-sol frontier

openai/gpt-5.6-sol

97.5

median score

30 attempts · bubbles above, range below

Mean
97.3
SD
1.09
IQR
1.5
USD / question
0.0021
Tokens / question
322
USD / 1M tok
6.58
Time / question
3.5 s
USD / test
0.376

by subject median across 30 sits

  • coding 97.7 92.1–100.0 · sd 1.9
  • data analysis 92.7 85.4–96.3 · sd 2.8
  • logic 97.6 95.3–100.0 · sd 1.3
  • perception 100.0 98.6–100.0 · sd 0.6
  • research 100.0 97.3–100.0 · sd 0.8

04

claude-fable-5.1 frontier

anthropic/claude-fable-5.1

96.9

median score

10 attempts · bubbles above, range below

Mean
96.8
SD
0.92
IQR
1.6
USD / question
0.019
Tokens / question
553
USD / 1M tok
35.16
Time / question
6.1 s
USD / test
3.44

by subject median across 10 sits

  • coding 97.7 93.4–100.0 · sd 2.1
  • data analysis 100.0 98.8–100.0 · sd 0.4
  • logic 97.6 91.8–100.0 · sd 2.3
  • perception 87.8 87.8–87.8 · sd 0.0
  • research 100.0 97.3–100.0 · sd 0.9

05

claude-opus-4.7 frontier

anthropic/claude-opus-4.7

96.4

median score

5 attempts · bubbles above, range below

Mean
95.7
SD
1.85
IQR
3.5
USD / question
0.0037
Tokens / question
316
USD / 1M tok
11.93
Time / question
1.9 s
USD / test
0.664

by subject median across 5 sits

  • coding 97.8 89.7–100.0 · sd 4.6
  • data analysis 98.8 97.6–100.0 · sd 1.2
  • logic 97.6 96.5–98.8 · sd 1.2
  • perception 86.5 86.5–86.5 · sd 0.0
  • research 98.7 98.7–100.0 · sd 0.7

06

gpt-5.6-terra frontier

openai/gpt-5.6-terra

96.4

median score

5 attempts · bubbles above, range below

Mean
96.3
SD
0.36
IQR
0.6
USD / question
0.0029
Tokens / question
360
USD / 1M tok
8.20
Time / question
2.9 s
USD / test
0.522

by subject median across 5 sits

  • coding 95.2 92.0–95.2 · sd 1.7
  • data analysis 93.9 91.5–95.1 · sd 1.6
  • logic 97.6 96.5–100.0 · sd 1.3
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 100.0 98.7–100.0 · sd 0.7

07

gpt-6-astra frontier

openai/gpt-6-astra

96.3

median score

10 attempts · bubbles above, range below

Mean
96.1
SD
0.93
IQR
1.0
USD / question
0.0083
Tokens / question
276
USD / 1M tok
29.94
Time / question
3.6 s
USD / test
1.46

by subject median across 10 sits

  • coding 99.7 93.4–100.0 · sd 2.4
  • data analysis 95.1 91.5–97.6 · sd 2.1
  • logic 97.6 94.1–100.0 · sd 1.6
  • perception 86.5 86.5–86.5 · sd 0.0
  • research 100.0 100.0–100.0 · sd 0.0

08

gpt-5.6-luna-pro frontier

openai/gpt-5.6-luna-pro

96.2

median score

5 attempts · bubbles above, range below

Mean
97.0
SD
1.28
IQR
2.2
USD / question
0.0018
Tokens / question
3.9k
USD / 1M tok
0.471
Time / question
6.6 s
USD / test
0.321

by subject median across 5 sits

  • coding 95.5 95.0–100.0 · sd 2.5
  • data analysis 92.7 90.2–96.3 · sd 2.3
  • logic 97.6 95.3–98.8 · sd 1.8
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 98.7 97.3–100.0 · sd 1.1

09

gpt-5.6-sol-pro frontier

openai/gpt-5.6-sol-pro

96.0

median score

5 attempts · bubbles above, range below

Mean
96.3
SD
1.39
IQR
2.5
USD / question
0.012
Tokens / question
3.3k
USD / 1M tok
3.69
Time / question
6.7 s
USD / test
2.14

by subject median across 5 sits

  • coding 97.8 95.0–100.0 · sd 2.3
  • data analysis 91.5 90.2–96.3 · sd 2.4
  • logic 97.6 95.3–100.0 · sd 1.9
  • perception 93.2 93.2–93.2 · sd 0.0
  • research 98.7 97.3–100.0 · sd 0.9

10

grok-4.5 frontier

x-ai/grok-4.5

95.2

median score

5 attempts · bubbles above, range below

Mean
95.2
SD
1.28
IQR
2.4
USD / question
0.0065
Tokens / question
1.3k
USD / 1M tok
4.86
Time / question
9.4 s
USD / test
1.14

by subject median across 5 sits

  • coding 96.6 91.8–100.0 · sd 3.3
  • data analysis 100.0 98.8–100.0 · sd 0.7
  • logic 98.8 95.3–100.0 · sd 1.8
  • perception 81.1 81.1–81.1 · sd 0.0
  • research 98.7 96.0–98.7 · sd 1.2

11

claude-sonnet-4.6 frontier

anthropic/claude-sonnet-4.6

95.0

median score

11 attempts · bubbles above, range below

Mean
93.9
SD
4.56
IQR
3.5
USD / question
0.0026
Tokens / question
287
USD / 1M tok
9.06
Time / question
3.2 s
USD / test
0.285

by subject median across 11 sits

  • coding 96.4 92.9–100.0 · sd 2.3
  • data analysis 96.4 89.3–100.0 · sd 3.7
  • logic 82.1 28.6–100.0 · sd 18.2
  • perception 100.0 92.9–100.0 · sd 2.5
  • research 100.0 96.4–100.0 · sd 1.1

12

grok-4.7 frontier

x-ai/grok-4.7

94.7

median score

11 attempts · bubbles above, range below

Mean
94.5
SD
0.51
IQR
0.7
USD / question
0.014
Tokens / question
4.1k
USD / 1M tok
3.48
Time / question
10.9 s
USD / test
2.95

by subject median across 11 sits

  • coding 95.8 91.8–96.2 · sd 1.6
  • data analysis 100.0 98.9–100.0 · sd 0.5
  • logic 95.7 94.6–97.8 · sd 1.0
  • perception 82.1 82.1–82.1 · sd 0.0
  • research 98.8 97.6–98.8 · sd 0.6

13

grok-4.3 frontier

x-ai/grok-4.3

94.7

median score

5 attempts · bubbles above, range below

Mean
95.0
SD
1.41
IQR
2.4
USD / question
0.0017
Tokens / question
934
USD / 1M tok
1.85
Time / question
5.5 s
USD / test
0.306

by subject median across 5 sits

  • coding 92.3 87.5–97.3 · sd 3.5
  • data analysis 98.8 98.8–100.0 · sd 0.7
  • logic 97.6 94.1–100.0 · sd 2.1
  • perception 87.8 87.8–87.8 · sd 0.0
  • research 100.0 100.0–100.0 · sd 0.0

14

claude-opus-4.8 frontier

anthropic/claude-opus-4.8

94.6

median score

5 attempts · bubbles above, range below

Mean
95.3
SD
1.54
IQR
2.8
USD / question
0.0053
Tokens / question
375
USD / 1M tok
14.27
Time / question
2.8 s
USD / test
0.946

by subject median across 5 sits

  • coding 97.8 92.4–100.0 · sd 3.2
  • data analysis 95.1 93.9–98.8 · sd 1.9
  • logic 98.8 96.5–100.0 · sd 1.5
  • perception 85.1 83.8–86.5 · sd 1.1
  • research 98.7 94.7–98.7 · sd 1.7

15

gpt-5.6-terra-pro frontier

openai/gpt-5.6-terra-pro

94.3

median score

5 attempts · bubbles above, range below

Mean
94.6
SD
2.02
IQR
3.8
USD / question
0.015
Tokens / question
3.4k
USD / 1M tok
4.29
Time / question
8.0 s
USD / test
2.62

by subject median across 5 sits

  • coding 92.2 84.8–95.9 · sd 4.6
  • data analysis 90.2 87.8–96.3 · sd 3.2
  • logic 96.5 95.3–98.8 · sd 1.6
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 98.7 96.0–98.7 · sd 1.2

16

gpt-4.1-mini flash

openai/gpt-4.1-mini

94.2

median score

5 attempts · bubbles above, range below

Mean
94.0
SD
0.72
IQR
1.1
USD / question
0.0006
Tokens / question
457
USD / 1M tok
1.22
Time / question
2.6 s
USD / test
0.073

by subject median across 5 sits

  • coding 91.8 89.3–94.8 · sd 2.2
  • data analysis 95.2 95.2–95.2 · sd 0.0
  • logic 96.0 90.0–98.0 · sd 3.3
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

17

gpt-5.6-luna frontier

openai/gpt-5.6-luna

94.2

median score

5 attempts · bubbles above, range below

Mean
94.3
SD
0.53
IQR
1.0
USD / question
0.0003
Tokens / question
401
USD / 1M tok
0.856
Time / question
3.0 s
USD / test
0.061

by subject median across 5 sits

  • coding 95.2 94.4–97.3 · sd 1.4
  • data analysis 95.1 93.9–97.6 · sd 1.6
  • logic 91.8 91.8–92.9 · sd 0.6
  • perception 86.5 86.5–86.5 · sd 0.0
  • research 100.0 98.7–100.0 · sd 0.7

18

claude-opus-5.5 frontier

anthropic/claude-opus-5.5

94.0

median score

16 attempts · bubbles above, range below

Mean
93.7
SD
1.59
IQR
1.4
USD / question
0.010
Tokens / question
674
USD / 1M tok
15.11
Time / question
6.1 s
USD / test
2.11

by subject median across 16 sits

  • coding 93.3 88.0–98.5 · sd 2.7
  • data analysis 100.0 97.7–100.0 · sd 0.7
  • logic 97.8 90.2–98.9 · sd 2.2
  • perception 82.1 79.5–82.1 · sd 0.6
  • research 96.3 87.8–98.8 · sd 2.7

19

gpt-5.5 frontier

openai/gpt-5.5

94.0

median score

5 attempts · bubbles above, range below

Mean
94.0
SD
0.75
IQR
1.2
USD / question
0.0098
Tokens / question
443
USD / 1M tok
22.23
Time / question
3.9 s
USD / test
1.74

by subject median across 5 sits

  • coding 92.0 90.9–95.2 · sd 1.7
  • data analysis 92.7 91.5–93.9 · sd 1.0
  • logic 98.8 96.5–100.0 · sd 1.3
  • perception 87.8 87.8–87.8 · sd 0.0
  • research 100.0 100.0–100.0 · sd 0.0

20

claude-sonnet-5.5 frontier

anthropic/claude-sonnet-5.5

93.9

median score

10 attempts · bubbles above, range below

Mean
93.7
SD
1.84
IQR
3.5
USD / question
0.0047
Tokens / question
642
USD / 1M tok
7.36
Time / question
3.4 s
USD / test
0.625

by subject median across 10 sits

  • coding 96.8 91.5–100.0 · sd 3.4
  • data analysis 100.0 100.0–100.0 · sd 0.0
  • logic 97.0 92.0–100.0 · sd 2.8
  • perception 69.0 69.0–69.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

21

claude-sonnet-5 frontier

anthropic/claude-sonnet-5

93.8

median score

5 attempts · bubbles above, range below

Mean
93.9
SD
0.66
IQR
1.2
USD / question
0.0034
Tokens / question
496
USD / 1M tok
6.78
Time / question
4.6 s
USD / test
0.595

by subject median across 5 sits

  • coding 94.4 92.0–95.2 · sd 1.5
  • data analysis 91.5 91.5–92.7 · sd 0.7
  • logic 97.6 96.5–100.0 · sd 1.3
  • perception 86.5 86.5–86.5 · sd 0.0
  • research 98.7 97.3–100.0 · sd 1.1

22

claude-sonnet-4 frontier

anthropic/claude-sonnet-4

93.6

median score

5 attempts · bubbles above, range below

Mean
93.4
SD
1.32
IQR
2.2
USD / question
0.0075
Tokens / question
626
USD / 1M tok
12.02
Time / question
5.1 s
USD / test
0.993

by subject median across 5 sits

  • coding 94.2 89.3–96.7 · sd 2.9
  • data analysis 100.0 100.0–100.0 · sd 0.0
  • logic 96.0 94.0–100.0 · sd 2.3
  • perception 79.3 79.3–82.8 · sd 1.5
  • research 96.6 96.6–96.6 · sd 0.0

23

gpt-6-astra-pro frontier

openai/gpt-6-astra-pro

93.2

median score

5 attempts · bubbles above, range below

Mean
93.8
SD
1.45
IQR
2.6
USD / question
0.036
Tokens / question
2.4k
USD / 1M tok
15.04
Time / question
5.6 s
USD / test
6.40

by subject median across 5 sits

  • coding 97.8 95.0–100.0 · sd 2.3
  • data analysis 90.2 86.6–95.1 · sd 3.3
  • logic 97.6 94.1–100.0 · sd 2.3
  • perception 81.1 81.1–81.1 · sd 0.0
  • research 97.3 93.3–98.7 · sd 2.0

24

o4-mini frontier

openai/o4-mini

92.9

median score

11 attempts · bubbles above, range below

Mean
91.7
SD
3.99
IQR
1.5
USD / question
0.0026
Tokens / question
694
USD / 1M tok
3.80
Time / question
5.1 s
USD / test
0.288

by subject median across 11 sits

  • coding 96.4 92.9–100.0 · sd 2.3
  • data analysis 92.9 85.7–100.0 · sd 5.4
  • logic 75.0 28.6–100.0 · sd 17.7
  • perception 96.4 85.7–100.0 · sd 4.5
  • research 100.0 89.3–100.0 · sd 3.3

25

kimi-k3 frontier

moonshotai/kimi-k3

92.8

median score

38 attempts · bubbles above, range below

Mean
92.1
SD
2.55
IQR
2.3
USD / question
0.0088
Tokens / question
819
USD / 1M tok
10.73
Time / question
8.5 s
USD / test
1.56

by subject median across 38 sits

  • coding 90.8 64.1–94.8 · sd 5.2
  • data analysis 97.6 86.4–100.0 · sd 2.8
  • logic 96.5 90.2–100.0 · sd 2.2
  • perception 81.1 76.9–82.1 · sd 0.7
  • research 98.7 91.5–100.0 · sd 1.8

26

qwen3.7-plus open

qwen/qwen3.7-plus

91.6

median score

5 attempts · bubbles above, range below

Mean
91.4
SD
0.84
IQR
1.5
USD / question
0.0014
Tokens / question
1.2k
USD / 1M tok
1.16
Time / question
13.4 s
USD / test
0.247

by subject median across 5 sits

  • coding 89.6 87.0–96.5 · sd 3.7
  • data analysis 91.5 86.6–92.7 · sd 2.4
  • logic 96.5 96.5–98.8 · sd 1.1
  • perception 81.1 78.4–81.1 · sd 1.2
  • research 98.7 94.7–98.7 · sd 1.7

27

qwen3.7-max open

qwen/qwen3.7-max

91.4

median score

5 attempts · bubbles above, range below

Mean
90.5
SD
1.67
IQR
3.1
USD / question
0.0049
Tokens / question
1.2k
USD / 1M tok
4.07
Time / question
11.5 s
USD / test
0.873

by subject median across 5 sits

  • coding 87.4 80.4–88.9 · sd 3.8
  • data analysis 100.0 97.6–100.0 · sd 1.3
  • logic 98.8 95.3–98.8 · sd 1.7
  • perception 75.7 75.7–75.7 · sd 0.0
  • research 100.0 98.7–100.0 · sd 0.7

28

gpt-5.4 frontier

openai/gpt-5.4

90.7

median score

11 attempts · bubbles above, range below

Mean
90.1
SD
4.21
IQR
2.1
USD / question
0.0023
Tokens / question
259
USD / 1M tok
8.66
Time / question
2.1 s
USD / test
0.249

by subject median across 11 sits

  • coding 100.0 96.4–100.0 · sd 1.8
  • data analysis 96.4 89.3–100.0 · sd 3.7
  • logic 64.3 14.3–96.4 · sd 19.6
  • perception 96.4 85.7–100.0 · sd 4.4
  • research 96.4 92.9–100.0 · sd 2.3

29

gpt-6.1-sol frontier

openai/gpt-6.1-sol

88.6

median score

10 attempts · bubbles above, range below

Mean
88.3
SD
1.67
IQR
3.3
USD / question
0.0022
Tokens / question
333
USD / 1M tok
6.54
Time / question
4.0 s
USD / test
0.288

by subject median across 10 sits

  • coding 97.2 92.3–100.0 · sd 3.2
  • data analysis 4.8 4.8–4.8 · sd 0.0
  • logic 97.0 92.0–100.0 · sd 2.8
  • perception 86.2 86.2–86.2 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

30

gpt-6-luna-pro frontier

openai/gpt-6-luna-pro

88.5

median score

10 attempts · bubbles above, range below

Mean
88.9
SD
1.16
IQR
1.6
USD / question
0.0007
Tokens / question
3.3k
USD / 1M tok
0.203
Time / question
6.6 s
USD / test
0.089

by subject median across 10 sits

  • coding 94.4 93.8–100.0 · sd 2.1
  • data analysis 4.8 4.8–4.8 · sd 0.0
  • logic 94.0 88.0–98.0 · sd 3.2
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

31

ember-1 standard

fireworks/ember-1

88.2

median score

10 attempts · bubbles above, range below

Mean
88.0
SD
1.49
IQR
2.4
USD / question
0.010
Tokens / question
860
USD / 1M tok
11.73
Time / question
7.7 s
USD / test
1.33

by subject median across 10 sits

  • coding 95.9 92.6–100.0 · sd 2.2
  • data analysis 42.9 42.9–47.6 · sd 2.5
  • logic 92.0 84.0–94.0 · sd 3.3
  • perception 69.0 69.0–69.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

32

gpt-6-sol frontier

openai/gpt-6-sol

88.2

median score

10 attempts · bubbles above, range below

Mean
87.9
SD
1.70
IQR
3.4
USD / question
0.0023
Tokens / question
346
USD / 1M tok
6.67
Time / question
3.3 s
USD / test
0.305

by subject median across 10 sits

  • coding 97.2 92.3–100.0 · sd 3.2
  • data analysis 4.8 4.8–4.8 · sd 0.0
  • logic 97.0 92.0–100.0 · sd 2.8
  • perception 82.8 82.8–82.8 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

33

qwen3-max open

qwen/qwen3-max

87.9

median score

11 attempts · bubbles above, range below

Mean
86.4
SD
3.93
IQR
2.9
USD / question
0.0013
Tokens / question
436
USD / 1M tok
2.93
Time / question
5.2 s
USD / test
0.140

by subject median across 11 sits

  • coding 82.1 78.6–82.1 · sd 1.9
  • data analysis 96.4 89.3–100.0 · sd 3.7
  • logic 78.6 28.6–85.7 · sd 16.0
  • perception 85.7 78.6–85.7 · sd 2.5
  • research 100.0 100.0–100.0 · sd 0.0

34

glm-5.3-flash flash

z-ai/glm-5.3-flash

87.7

median score

30 attempts · bubbles above, range below

Mean
87.7
SD
1.70
IQR
2.1
USD / question
0.0003
Tokens / question
833
USD / 1M tok
0.416
Time / question
6.5 s
USD / test
0.062

by subject median across 30 sits

  • coding 79.7 70.4–88.2 · sd 4.3
  • data analysis 95.1 89.0–97.6 · sd 2.4
  • logic 96.5 91.8–98.8 · sd 1.6
  • perception 81.1 79.7–81.1 · sd 0.6
  • research 98.7 97.3–100.0 · sd 0.8

35

inkling standard

thinkingmachines/inkling

87.5

median score

30 attempts · bubbles above, range below

Mean
87.8
SD
1.71
IQR
2.4
USD / question
0.0025
Tokens / question
718
USD / 1M tok
3.43
Time / question
7.6 s
USD / test
0.435

by subject median across 30 sits

  • coding 78.5 69.7–88.9 · sd 4.4
  • data analysis 92.7 90.2–93.9 · sd 1.0
  • logic 96.5 94.1–100.0 · sd 1.6
  • perception 81.1 81.1–81.1 · sd 0.0
  • research 100.0 97.3–100.0 · sd 0.8

36

gpt-6-sol-pro frontier

openai/gpt-6-sol-pro

87.5

median score

10 attempts · bubbles above, range below

Mean
87.8
SD
1.09
IQR
1.8
USD / question
0.0089
Tokens / question
2.6k
USD / 1M tok
3.38
Time / question
5.0 s
USD / test
1.18

by subject median across 10 sits

  • coding 96.6 94.1–100.0 · sd 1.9
  • data analysis 4.8 4.8–4.8 · sd 0.0
  • logic 94.0 88.0–98.0 · sd 3.2
  • perception 82.8 82.8–82.8 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

37

claude-haiku-5.5 flash

anthropic/claude-haiku-5.5

86.7

median score

5 attempts · bubbles above, range below

Mean
86.9
SD
1.18
IQR
2.0
USD / question
0.0003
Tokens / question
859
USD / 1M tok
0.401
Time / question
3.1 s
USD / test
0.045

by subject median across 5 sits

  • coding 87.4 84.6–90.5 · sd 2.5
  • data analysis 100.0 100.0–100.0 · sd 0.0
  • logic 86.0 84.0–88.0 · sd 1.7
  • perception 69.0 69.0–69.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

38

gpt-5.4-nano flash

openai/gpt-5.4-nano

86.7

median score

5 attempts · bubbles above, range below

Mean
86.1
SD
1.43
IQR
2.5
USD / question
0.0004
Tokens / question
408
USD / 1M tok
0.879
Time / question
2.0 s
USD / test
0.047

by subject median across 5 sits

  • coding 91.1 87.2–97.0 · sd 3.6
  • data analysis 61.9 61.9–66.7 · sd 2.6
  • logic 84.0 76.0–92.0 · sd 6.1
  • perception 75.9 75.9–86.2 · sd 4.5
  • research 89.7 75.9–96.6 · sd 7.9

39

mistral-medium-3.1 open

mistralai/mistral-medium-3.1

86.6

median score

30 attempts · bubbles above, range below

Mean
86.6
SD
1.19
IQR
2.1
USD / question
0.0007
Tokens / question
472
USD / 1M tok
1.49
Time / question
1.6 s
USD / test
0.124

by subject median across 30 sits

  • coding 94.2 87.6–98.8 · sd 3.4
  • data analysis 89.0 84.1–91.5 · sd 1.8
  • logic 50.6 48.2–54.1 · sd 1.4
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 94.7 90.7–94.7 · sd 0.9

40

gpt-6.1-sol-pro frontier

openai/gpt-6.1-sol-pro

86.5

median score

10 attempts · bubbles above, range below

Mean
86.6
SD
1.11
IQR
1.8
USD / question
0.0082
Tokens / question
2.6k
USD / 1M tok
3.19
Time / question
6.7 s
USD / test
1.09

by subject median across 10 sits

  • coding 96.6 94.1–100.0 · sd 1.9
  • data analysis 4.8 4.8–4.8 · sd 0.0
  • logic 96.0 90.0–100.0 · sd 3.2
  • perception 69.0 69.0–69.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

41

deepseek-v4-pro open

deepseek/deepseek-v4-pro

85.7

median score

5 attempts · bubbles above, range below

Mean
85.5
SD
2.76
IQR
5.3
USD / question
0.0018
Tokens / question
828
USD / 1M tok
2.14
Time / question
9.8 s
USD / test
0.315

by subject median across 5 sits

  • coding 71.5 62.5–77.8 · sd 6.1
  • data analysis 98.8 97.6–100.0 · sd 1.0
  • logic 94.1 90.6–97.6 · sd 2.8
  • perception 81.1 79.7–81.1 · sd 0.7
  • research 100.0 100.0–100.0 · sd 0.0

42

command-a open

cohere/command-a

85.1

median score

30 attempts · bubbles above, range below

Mean
84.5
SD
1.80
IQR
2.6
USD / question
0.0028
Tokens / question
376
USD / 1M tok
7.32
Time / question
4.4 s
USD / test
0.487

by subject median across 30 sits

  • coding 78.1 67.7–86.4 · sd 4.7
  • data analysis 96.3 93.9–100.0 · sd 2.0
  • logic 72.9 68.2–76.5 · sd 2.1
  • perception 87.8 86.5–87.8 · sd 0.5
  • research 99.3 96.0–100.0 · sd 1.1

43

claude-opus-5 frontier

anthropic/claude-opus-5

85.1

median score

15 attempts · bubbles above, range below

Mean
85.2
SD
1.55
IQR
2.3
USD / question
0.017
Tokens / question
830
USD / 1M tok
20.11
Time / question
6.1 s
USD / test
2.96

by subject median across 15 sits

  • coding 64.0 57.7–72.2 · sd 4.0
  • data analysis 100.0 98.8–100.0 · sd 0.4
  • logic 98.8 96.5–100.0 · sd 1.1
  • perception 93.2 91.9–93.2 · sd 0.6
  • research 100.0 96.0–100.0 · sd 1.1

44

grok-4.20 frontier

x-ai/grok-4.20

84.9

median score

10 attempts · bubbles above, range below

Mean
85.0
SD
2.99
IQR
5.8
USD / question
0.0009
Tokens / question
581
USD / 1M tok
1.52
Time / question
2.9 s
USD / test
0.116

by subject median across 10 sits

  • coding 87.0 81.2–95.8 · sd 4.6
  • data analysis 100.0 100.0–100.0 · sd 0.0
  • logic 77.0 72.0–84.0 · sd 4.0
  • perception 65.5 58.6–69.0 · sd 3.6
  • research 96.6 93.1–96.6 · sd 1.1

45

gpt-6-luna frontier

openai/gpt-6-luna

84.8

median score

10 attempts · bubbles above, range below

Mean
85.2
SD
1.47
IQR
2.0
USD / question
0.0002
Tokens / question
485
USD / 1M tok
0.381
Time / question
3.5 s
USD / test
0.024

by subject median across 10 sits

  • coding 90.1 87.4–96.5 · sd 2.7
  • data analysis 4.8 4.8–4.8 · sd 0.0
  • logic 88.0 84.0–92.0 · sd 2.9
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

46

gpt-4.1 standard

openai/gpt-4.1

84.5

median score

5 attempts · bubbles above, range below

Mean
85.0
SD
1.27
IQR
2.0
USD / question
0.0028
Tokens / question
455
USD / 1M tok
6.07
Time / question
2.6 s
USD / test
0.364

by subject median across 5 sits

  • coding 88.6 87.6–94.8 · sd 2.9
  • data analysis 85.7 85.7–85.7 · sd 0.0
  • logic 66.0 64.0–66.0 · sd 1.1
  • perception 82.8 82.8–82.8 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

47

mistral-large-4-0 open

mistralai/mistral-large-4-0

84.3

median score

10 attempts · bubbles above, range below

Mean
84.3
SD
2.39
IQR
2.3
USD / question
0.0008
Tokens / question
492
USD / 1M tok
1.65
Time / question
3.6 s
USD / test
0.107

by subject median across 10 sits

  • coding 84.7 76.8–93.0 · sd 4.5
  • data analysis 64.3 61.9–66.7 · sd 2.5
  • logic 95.0 86.0–98.0 · sd 3.8
  • perception 69.0 69.0–69.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

48

devstral-2512 standard

mistralai/devstral-2512

83.6

median score

30 attempts · bubbles above, range below

Mean
83.4
SD
1.59
IQR
2.6
USD / question
0.0007
Tokens / question
475
USD / 1M tok
1.49
Time / question
1.5 s
USD / test
0.125

by subject median across 30 sits

  • coding 93.0 84.7–98.8 · sd 4.2
  • data analysis 89.0 84.1–91.5 · sd 1.9
  • logic 47.1 43.5–49.4 · sd 1.3
  • perception 86.5 86.5–86.5 · sd 0.0
  • research 94.7 92.0–94.7 · sd 0.7

49

inkling-small flash

thinkingmachines/inkling-small

82.1

median score

30 attempts · bubbles above, range below

Mean
82.3
SD
2.01
IQR
3.3
USD / question
0.0008
Tokens / question
756
USD / 1M tok
1.06
Time / question
2.4 s
USD / test
0.142

by subject median across 30 sits

  • coding 62.9 54.3–73.0 · sd 5.3
  • data analysis 93.9 89.0–96.3 · sd 2.0
  • logic 96.5 92.9–98.8 · sd 1.8
  • perception 81.1 81.1–81.1 · sd 0.0
  • research 98.7 98.7–100.0 · sd 0.7

50

gpt-5.5-pro frontier

openai/gpt-5.5-pro

80.9

median score

5 attempts · bubbles above, range below

Mean
81.4
SD
1.73
IQR
3.1
USD / question
0.104
Tokens / question
690
USD / 1M tok
150.56
Time / question
13.1 s
USD / test
18.36

by subject median across 5 sits

  • coding 63.7 58.7–66.5 · sd 2.9
  • data analysis 90.2 85.4–95.1 · sd 4.4
  • logic 95.3 94.1–100.0 · sd 2.3
  • perception 81.1 81.1–81.1 · sd 0.0
  • research 100.0 98.7–100.0 · sd 0.7

51

glm-5.3 standard

z-ai/glm-5.3

80.8

median score

33 attempts · bubbles above, range below

Mean
80.6
SD
1.99
IQR
1.7
USD / question
0.0032
Tokens / question
894
USD / 1M tok
3.57
Time / question
4.9 s
USD / test
0.570

by subject median across 33 sits

  • coding 64.0 51.6–71.4 · sd 4.3
  • data analysis 92.7 86.4–98.9 · sd 2.0
  • logic 94.1 84.8–97.6 · sd 2.9
  • perception 81.1 76.9–88.5 · sd 1.6
  • research 92.0 89.3–95.1 · sd 1.3

52

codestral-2508 standard

mistralai/codestral-2508

79.5

median score

30 attempts · bubbles above, range below

Mean
79.5
SD
1.71
IQR
2.5
USD / question
0.0003
Tokens / question
377
USD / 1M tok
0.679
Time / question
1.8 s
USD / test
0.045

by subject median across 30 sits

  • coding 89.3 81.1–95.2 · sd 4.0
  • data analysis 51.8 47.6–56.1 · sd 2.3
  • logic 80.0 76.5–85.9 · sd 2.5
  • perception 83.8 79.7–87.8 · sd 2.0
  • research 82.7 77.3–88.0 · sd 2.9

53

llama-4-maverick open

meta-llama/llama-4-maverick

79.4

median score

30 attempts · bubbles above, range below

Mean
79.3
SD
1.34
IQR
1.9
USD / question
0.0004
Tokens / question
599
USD / 1M tok
0.637
Time / question
6.1 s
USD / test
0.068

by subject median across 30 sits

  • coding 93.7 86.6–98.9 · sd 3.8
  • data analysis 70.7 64.6–74.4 · sd 2.3
  • logic 92.9 84.7–97.6 · sd 2.9
  • perception 58.8 52.7–62.2 · sd 2.5
  • research 62.7 58.7–66.7 · sd 2.0

54

mimo-v2.6-pro standard

xiaomi/mimo-v2.6-pro

79.1

median score

10 attempts · bubbles above, range below

Mean
78.7
SD
0.97
IQR
1.8
USD / question
0.0007
Tokens / question
926
USD / 1M tok
0.803
Time / question
18.5 s
USD / test
0.155

by subject median across 10 sits

  • coding 66.7 62.9–69.4 · sd 2.5
  • data analysis 86.9 85.2–90.9 · sd 1.9
  • logic 91.8 90.2–94.6 · sd 1.6
  • perception 76.3 71.8–76.9 · sd 1.9
  • research 85.4 84.1–86.6 · sd 1.0

55

deepseek-v4-pro-0813 open

deepseek/deepseek-v4-pro-0813

77.7

median score

8 attempts · bubbles above, range below

Mean
76.7
SD
3.79
IQR
6.2
USD / question
0.0017
Tokens / question
763
USD / 1M tok
2.37
Time / question
6.8 s
USD / test
0.313

by subject median across 8 sits

  • coding 54.3 47.4–65.9 · sd 5.7
  • data analysis 97.0 84.1–100.0 · sd 5.8
  • logic 85.9 68.5–89.4 · sd 8.2
  • perception 81.1 79.7–88.5 · sd 3.2
  • research 99.4 92.7–100.0 · sd 2.6

56

mistral-medium-3-5 open

mistralai/mistral-medium-3-5

77.5

median score

10 attempts · bubbles above, range below

Mean
78.7
SD
2.90
IQR
4.9
USD / question
0.0033
Tokens / question
568
USD / 1M tok
5.83
Time / question
1.7 s
USD / test
0.437

by subject median across 10 sits

  • coding 79.5 75.6–89.8 · sd 5.1
  • data analysis 38.1 38.1–42.9 · sd 2.5
  • logic 65.0 60.0–68.0 · sd 2.7
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

57

mimo-v2.5-pro standard

xiaomi/mimo-v2.5-pro

77.1

median score

5 attempts · bubbles above, range below

Mean
77.0
SD
2.13
IQR
3.7
USD / question
0.0008
Tokens / question
1.1k
USD / 1M tok
0.752
Time / question
17.7 s
USD / test
0.107

by subject median across 5 sits

  • coding 67.1 61.1–73.0 · sd 4.7
  • data analysis 90.5 90.5–90.5 · sd 0.0
  • logic 94.0 90.0–98.0 · sd 3.4
  • perception 65.5 58.6–69.0 · sd 4.2
  • research 96.6 93.1–96.6 · sd 1.5

58

deepseek-v4.1-flash flash

deepseek/deepseek-v4.1-flash

76.9

median score

5 attempts · bubbles above, range below

Mean
76.6
SD
1.06
IQR
2.0
USD / question
0.0006
Tokens / question
748
USD / 1M tok
0.801
Time / question
4.7 s
USD / test
0.106

by subject median across 5 sits

  • coding 47.0 45.7–49.2 · sd 1.6
  • data analysis 95.1 93.9–98.8 · sd 2.0
  • logic 96.5 91.8–96.5 · sd 2.3
  • perception 81.1 81.1–81.1 · sd 0.0
  • research 100.0 100.0–100.0 · sd 0.0

59

llama-3.3-70b-instruct open

meta-llama/llama-3.3-70b-instruct

76.2

median score

5 attempts · bubbles above, range below

Mean
74.8
SD
3.10
IQR
5.6
USD / question
0.0002
Tokens / question
526
USD / 1M tok
0.421
Time / question
10.7 s
USD / test
0.029

by subject median across 5 sits

  • coding 73.7 62.3–74.8 · sd 5.6
  • data analysis 85.7 85.7–85.7 · sd 0.0
  • logic 56.0 54.0–60.0 · sd 2.6
  • perception 100.0 100.0–100.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

60

deepseek-r1-0528 open

deepseek/deepseek-r1-0528

73.6

median score

11 attempts · bubbles above, range below

Mean
72.5
SD
5.43
IQR
2.9
USD / question
0.0044
Tokens / question
2.1k
USD / 1M tok
2.10
Time / question
62.1 s
USD / test
0.489

by subject median across 11 sits

  • coding 39.3 35.7–50.0 · sd 4.0
  • data analysis 60.7 42.9–67.9 · sd 6.7
  • logic 78.6 28.6–96.4 · sd 18.0
  • perception 85.7 78.6–100.0 · sd 5.5
  • research 100.0 96.4–100.0 · sd 1.4

61

gemini-3-flash-preview flash

google/gemini-3-flash-preview

73.3

median score

30 attempts · bubbles above, range below

Mean
76.6
SD
9.57
IQR
4.0
USD / question
0.0006
Tokens / question
286
USD / 1M tok
2.21
Time / question
3.6 s
USD / test
0.064

by subject median across 30 sits

  • coding 100.0 100.0–100.0 · sd 0.0
  • data analysis 69.6 53.6–100.0 · sd 13.4
  • logic 71.4 57.1–100.0 · sd 11.6
  • perception 71.4 57.1–100.0 · sd 12.0
  • research 67.9 46.4–100.0 · sd 14.6

62

mimo-v2.6-flash flash

xiaomi/mimo-v2.6-flash

73.2

median score

10 attempts · bubbles above, range below

Mean
73.2
SD
2.21
IQR
3.0
USD / question
0.0002
Tokens / question
860
USD / 1M tok
0.262
Time / question
13.7 s
USD / test
0.030

by subject median across 10 sits

  • coding 72.1 63.1–76.5 · sd 3.8
  • data analysis 23.8 19.0–23.8 · sd 2.0
  • logic 93.0 86.0–96.0 · sd 3.4
  • perception 65.5 58.6–69.0 · sd 3.2
  • research 93.1 89.7–96.6 · sd 2.8

63

deepseek-v4-flash flash

deepseek/deepseek-v4-flash

72.5

median score

30 attempts · bubbles above, range below

Mean
75.3
SD
9.15
IQR
6.1
USD / question
0.0001
Tokens / question
391
USD / 1M tok
0.230
Time / question
7.6 s
USD / test
0.0091

by subject median across 30 sits

  • coding 94.7 94.7–94.7 · sd 0.0
  • data analysis 75.0 53.6–100.0 · sd 12.6
  • logic 69.6 53.6–100.0 · sd 12.0
  • perception 60.7 42.9–85.7 · sd 11.8
  • research 75.0 53.6–100.0 · sd 13.2

64

mimo-v2.5 standard

xiaomi/mimo-v2.5

71.6

median score

5 attempts · bubbles above, range below

Mean
73.1
SD
2.81
IQR
4.8
USD / question
0.0002
Tokens / question
822
USD / 1M tok
0.300
Time / question
11.8 s
USD / test
0.032

by subject median across 5 sits

  • coding 71.1 70.5–81.6 · sd 4.8
  • data analysis 14.3 9.5–14.3 · sd 2.1
  • logic 90.0 88.0–92.0 · sd 1.7
  • perception 65.5 62.1–65.5 · sd 1.5
  • research 89.7 86.2–93.1 · sd 2.9

65

gemini-2.5-pro frontier

google/gemini-2.5-pro

71.4

median score

11 attempts · bubbles above, range below

Mean
69.8
SD
3.84
IQR
1.4
USD / question
0.022
Tokens / question
2.3k
USD / 1M tok
9.50
Time / question
20.7 s
USD / test
2.43

by subject median across 11 sits

  • coding 0.0 0.0–3.6 · sd 1.1
  • data analysis 92.9 89.3–100.0 · sd 3.9
  • logic 75.0 28.6–89.3 · sd 16.3
  • perception 82.1 78.6–85.7 · sd 3.1
  • research 100.0 92.9–100.0 · sd 2.1

66

gemini-3.1-flash-lite flash

google/gemini-3.1-flash-lite

70.6

median score

30 attempts · bubbles above, range below

Mean
73.9
SD
9.27
IQR
4.0
USD / question
0.0003
Tokens / question
285
USD / 1M tok
1.09
Time / question
2.0 s
USD / test
0.031

by subject median across 30 sits

  • coding 73.7 73.7–73.7 · sd 0.0
  • data analysis 67.9 53.6–100.0 · sd 13.1
  • logic 73.2 57.1–100.0 · sd 11.8
  • perception 75.0 57.1–100.0 · sd 11.8
  • research 67.9 50.0–100.0 · sd 13.5

67

deepseek-v4-flash-0731 flash

deepseek/deepseek-v4-flash-0731

70.2

median score

30 attempts · bubbles above, range below

Mean
72.7
SD
8.35
IQR
5.9
USD / question
0.0001
Tokens / question
699
USD / 1M tok
0.186
Time / question
5.0 s
USD / test
0.015

by subject median across 30 sits

  • coding 68.4 68.4–68.4 · sd 0.0
  • data analysis 75.0 53.6–100.0 · sd 12.1
  • logic 69.6 50.0–96.4 · sd 12.4
  • perception 60.7 46.4–85.7 · sd 10.4
  • research 82.1 60.7–100.0 · sd 11.7

68

qwen3.8-2.4t-a95b open

qwen/qwen3.8-2.4t-a95b

70.2

median score

5 attempts · bubbles above, range below

Mean
70.9
SD
1.81
IQR
2.6
USD / question
0.0052
Tokens / question
989
USD / 1M tok
5.23
Time / question
7.2 s
USD / test
0.917

by subject median across 5 sits

  • coding 30.9 29.3–39.8 · sd 4.1
  • data analysis 96.3 91.5–100.0 · sd 3.3
  • logic 95.3 92.9–96.5 · sd 1.6
  • perception 81.1 81.1–81.1 · sd 0.0
  • research 97.3 94.7–100.0 · sd 1.9

69

muse-spark-1.3 standard

meta/muse-spark-1.3

70.1

median score

6 attempts · bubbles above, range below

Mean
59.2
SD
29.06
IQR
20.5
USD / question
0.0053
Tokens / question
1.3k
USD / 1M tok
3.91
Time / question
10.1 s
USD / test
0.933

by subject median across 6 sits

  • coding 31.1 0.0–40.4 · sd 14.0
  • data analysis 94.5 0.0–97.6 · sd 38.9
  • logic 96.5 0.0–98.8 · sd 39.6
  • perception 80.4 0.0–81.1 · sd 32.9
  • research 97.3 0.0–98.7 · sd 39.8

70

gemini-3.7-flash flash

google/gemini-3.7-flash

69.8

median score

30 attempts · bubbles above, range below

Mean
72.5
SD
9.75
IQR
5.2
USD / question
0.0014
Tokens / question
445
USD / 1M tok
3.14
Time / question
5.5 s
USD / test
0.142

by subject median across 30 sits

  • coding 100.0 100.0–100.0 · sd 0.0
  • data analysis 69.6 53.6–100.0 · sd 13.4
  • logic 67.9 50.0–100.0 · sd 12.5
  • perception 55.4 42.9–85.7 · sd 12.3
  • research 67.9 46.4–100.0 · sd 14.6

71

gemini-3.8-flash flash

google/gemini-3.8-flash

69.5

median score

31 attempts · bubbles above, range below

Mean
72.8
SD
10.65
IQR
6.1
USD / question
0.0018
Tokens / question
536
USD / 1M tok
3.24
Time / question
5.4 s
USD / test
0.177

by subject median across 31 sits

  • coding 73.7 73.7–100.0 · sd 4.7
  • data analysis 71.4 53.6–100.0 · sd 14.1
  • logic 71.4 53.6–100.0 · sd 12.5
  • perception 71.4 57.1–100.0 · sd 13.1
  • research 67.9 46.4–100.0 · sd 15.4

72

kimi-k2 standard

moonshotai/kimi-k2

69.5

median score

30 attempts · bubbles above, range below

Mean
72.0
SD
9.53
IQR
4.0
USD / question
0.0012
Tokens / question
598
USD / 1M tok
2.02
Time / question
13.5 s
USD / test
0.122

by subject median across 30 sits

  • coding 94.7 94.7–94.7 · sd 0.0
  • data analysis 67.9 46.4–100.0 · sd 13.6
  • logic 69.6 53.6–100.0 · sd 12.3
  • perception 71.4 57.1–100.0 · sd 11.8
  • research 53.6 32.1–85.7 · sd 14.4

73

kimi-k2.5 standard

moonshotai/kimi-k2.5

69.5

median score

30 attempts · bubbles above, range below

Mean
72.0
SD
9.20
IQR
5.8
USD / question
0.0046
Tokens / question
2.1k
USD / 1M tok
2.23
Time / question
62.4 s
USD / test
0.464

by subject median across 30 sits

  • coding 78.9 78.9–78.9 · sd 0.0
  • data analysis 64.3 46.4–92.9 · sd 12.7
  • logic 71.4 50.0–100.0 · sd 11.7
  • perception 71.4 57.1–100.0 · sd 11.8
  • research 67.9 46.4–100.0 · sd 14.4

74

gemini-3.5-flash-lite flash

google/gemini-3.5-flash-lite

66.4

median score

30 attempts · bubbles above, range below

Mean
69.9
SD
9.41
IQR
4.8
USD / question
0.0006
Tokens / question
315
USD / 1M tok
1.86
Time / question
1.8 s
USD / test
0.060

by subject median across 30 sits

  • coding 78.9 78.9–78.9 · sd 0.0
  • data analysis 69.6 53.6–100.0 · sd 13.4
  • logic 69.6 50.0–92.9 · sd 11.8
  • perception 57.1 42.9–85.7 · sd 11.9
  • research 67.9 46.4–100.0 · sd 14.3

75

gemini-2.5-flash flash

google/gemini-2.5-flash

65.6

median score

30 attempts · bubbles above, range below

Mean
68.7
SD
9.70
IQR
5.5
USD / question
0.0009
Tokens / question
449
USD / 1M tok
2.05
Time / question
4.7 s
USD / test
0.093

by subject median across 30 sits

  • coding 68.4 68.4–68.4 · sd 0.0
  • data analysis 69.6 53.6–100.0 · sd 13.4
  • logic 71.4 50.0–100.0 · sd 12.4
  • perception 57.1 42.9–85.7 · sd 12.0
  • research 67.9 46.4–100.0 · sd 14.6

76

gemini-2.5-flash-lite flash

google/gemini-2.5-flash-lite

65.6

median score

30 attempts · bubbles above, range below

Mean
68.6
SD
9.03
IQR
6.3
USD / question
0.0002
Tokens / question
502
USD / 1M tok
0.342
Time / question
1.9 s
USD / test
0.017

by subject median across 30 sits

  • coding 78.9 78.9–78.9 · sd 0.0
  • data analysis 71.4 53.6–100.0 · sd 12.3
  • logic 67.9 42.9–96.4 · sd 11.4
  • perception 60.7 42.9–85.7 · sd 11.8
  • research 60.7 39.3–85.7 · sd 13.2

77

gemini-3.5-flash flash

google/gemini-3.5-flash

65.6

median score

30 attempts · bubbles above, range below

Mean
68.3
SD
9.74
IQR
4.4
USD / question
0.0066
Tokens / question
806
USD / 1M tok
8.16
Time / question
8.2 s
USD / test
0.662

by subject median across 30 sits

  • coding 73.7 73.7–73.7 · sd 0.0
  • data analysis 69.6 53.6–100.0 · sd 13.4
  • logic 69.6 50.0–100.0 · sd 12.1
  • perception 53.6 42.9–85.7 · sd 12.5
  • research 67.9 46.4–100.0 · sd 14.6

78

qwen3.8-flash flash

qwen/qwen3.8-flash

65.2

median score

30 attempts · bubbles above, range below

Mean
68.4
SD
9.09
IQR
4.5
USD / question
0.0003
Tokens / question
683
USD / 1M tok
0.407
Time / question
10.6 s
USD / test
0.028

by subject median across 30 sits

  • coding 63.2 63.2–63.2 · sd 0.0
  • data analysis 71.4 53.6–100.0 · sd 12.6
  • logic 67.9 50.0–92.9 · sd 11.6
  • perception 57.1 42.9–85.7 · sd 11.5
  • research 71.4 53.6–100.0 · sd 13.1

79

gemini-3.6-flash flash

google/gemini-3.6-flash

64.5

median score

30 attempts · bubbles above, range below

Mean
67.2
SD
9.53
IQR
5.4
USD / question
0.0027
Tokens / question
788
USD / 1M tok
3.40
Time / question
8.1 s
USD / test
0.272

by subject median across 30 sits

  • coding 68.4 68.4–68.4 · sd 0.0
  • data analysis 69.6 53.6–100.0 · sd 13.1
  • logic 69.6 50.0–100.0 · sd 12.3
  • perception 55.4 42.9–85.7 · sd 12.5
  • research 64.3 46.4–96.4 · sd 13.4

80

qwen3.5-flash-02-23 flash

qwen/qwen3.5-flash-02-23

63.8

median score

30 attempts · bubbles above, range below

Mean
66.7
SD
8.80
IQR
5.0
USD / question
0.0004
Tokens / question
1.8k
USD / 1M tok
0.248
Time / question
19.9 s
USD / test
0.044

by subject median across 30 sits

  • coding 100.0 100.0–100.0 · sd 0.0
  • data analysis 64.3 42.9–96.4 · sd 11.8
  • logic 39.3 21.4–67.9 · sd 11.7
  • perception 58.9 42.9–85.7 · sd 11.9
  • research 75.0 53.6–100.0 · sd 13.2

81

kimi-k2.6 standard

moonshotai/kimi-k2.6

61.8

median score

30 attempts · bubbles above, range below

Mean
64.0
SD
9.03
IQR
5.9
USD / question
0.0036
Tokens / question
1.2k
USD / 1M tok
3.00
Time / question
36.8 s
USD / test
0.366

by subject median across 30 sits

  • coding 78.9 78.9–78.9 · sd 0.0
  • data analysis 50.0 28.6–82.1 · sd 12.5
  • logic 64.3 46.4–100.0 · sd 11.5
  • perception 57.1 42.9–85.7 · sd 11.9
  • research 67.9 42.9–100.0 · sd 13.9

82

gemini-3.1-pro-preview frontier

google/gemini-3.1-pro-preview

58.0

median score

35 attempts · bubbles above, range below

Mean
58.2
SD
1.31
IQR
1.9
USD / question
0.013
Tokens / question
1.2k
USD / 1M tok
10.92
Time / question
9.3 s
USD / test
2.39

by subject median across 35 sits

  • coding 6.4 2.6–9.7 · sd 2.0
  • data analysis 85.4 80.5–87.8 · sd 1.7
  • logic 87.1 83.5–90.6 · sd 1.8
  • perception 81.1 81.1–81.1 · sd 0.0
  • research 100.0 98.7–100.0 · sd 0.6

83

mimo-v2.6-pro-ultraspeed standard

xiaomi/mimo-v2.6-pro-ultraspeed

56.4

median score

10 attempts · bubbles above, range below

Mean
55.6
SD
2.76
IQR
3.9
USD / question
0.0099
Tokens / question
1.2k
USD / 1M tok
8.15
Time / question
4.2 s
USD / test
1.31

by subject median across 10 sits

  • coding 40.8 30.2–44.3 · sd 5.2
  • data analysis 35.7 33.3–38.1 · sd 2.5
  • logic 93.0 86.0–98.0 · sd 3.3
  • perception 55.2 44.8–69.0 · sd 7.3
  • research 87.9 86.2–96.6 · sd 4.1

84

qwen3.6-flash flash

qwen/qwen3.6-flash

55.7

median score

30 attempts · bubbles above, range below

Mean
58.9
SD
9.12
IQR
6.8
USD / question
0.0016
Tokens / question
1.5k
USD / 1M tok
1.06
Time / question
15.1 s
USD / test
0.163

by subject median across 30 sits

  • coding 73.7 73.7–73.7 · sd 0.0
  • data analysis 50.0 32.1–78.6 · sd 12.7
  • logic 50.0 35.7–78.6 · sd 11.5
  • perception 58.9 42.9–85.7 · sd 11.9
  • research 58.9 35.7–85.7 · sd 13.6

85

qwen3.7-flash flash

qwen/qwen3.7-flash

54.6

median score

30 attempts · bubbles above, range below

Mean
56.9
SD
8.30
IQR
5.6
USD / question
0.0002
Tokens / question
1.7k
USD / 1M tok
0.123
Time / question
23.4 s
USD / test
0.021

by subject median across 30 sits

  • coding 78.9 78.9–78.9 · sd 0.0
  • data analysis 37.5 21.4–67.9 · sd 11.6
  • logic 48.2 35.7–75.0 · sd 10.7
  • perception 60.7 42.9–85.7 · sd 11.8
  • research 57.1 35.7–85.7 · sd 12.8

86

qwen3.8-max-0902 open

qwen/qwen3.8-max-0902

54.4

median score

13 attempts · bubbles above, range below

Mean
56.3
SD
7.06
IQR
9.9
USD / question
0.0056
Tokens / question
1.1k
USD / 1M tok
5.28
Time / question
10.6 s
USD / test
0.746

by subject median across 13 sits

  • coding 25.0 16.7–35.7 · sd 5.2
  • data analysis 90.5 89.8–97.7 · sd 2.7
  • logic 90.0 84.0–96.0 · sd 3.5
  • perception 69.0 69.0–82.1 · sd 5.1
  • research 96.6 86.6–96.6 · sd 2.8

87

muse-glimmer-30b standard

meta/muse-glimmer-30b

51.8

median score

10 attempts · bubbles above, range below

Mean
52.0
SD
2.20
IQR
4.1
USD / question
0.0014
Tokens / question
1.2k
USD / 1M tok
1.15
Time / question
16.3 s
USD / test
0.182

by subject median across 10 sits

  • coding 32.8 26.4–38.3 · sd 3.4
  • data analysis 14.3 9.5–14.3 · sd 2.0
  • logic 90.0 84.0–92.0 · sd 3.0
  • perception 69.0 65.5–69.0 · sd 1.7
  • research 96.6 96.6–96.6 · sd 0.0

88

command-a-plus open

cohere/command-a-plus

50.8

median score

10 attempts · bubbles above, range below

Mean
51.1
SD
2.28
IQR
3.5
USD / question
0.0020
Tokens / question
1.5k
USD / 1M tok
1.39
Time / question
10.2 s
USD / test
0.267

by subject median across 10 sits

  • coding 33.1 28.1–37.5 · sd 3.1
  • data analysis 42.9 42.9–47.6 · sd 2.3
  • logic 71.0 64.0–76.0 · sd 4.0
  • perception 69.0 65.5–69.0 · sd 1.4
  • research 93.1 89.7–96.6 · sd 2.3

89

qwen3.8-max-prime open

qwen/qwen3.8-max-prime

50.5

median score

10 attempts · bubbles above, range below

Mean
50.7
SD
1.88
IQR
3.4
USD / question
0.012
Tokens / question
1.1k
USD / 1M tok
10.60
Time / question
6.0 s
USD / test
1.54

by subject median across 10 sits

  • coding 17.1 13.3–22.4 · sd 3.2
  • data analysis 95.2 95.2–100.0 · sd 1.5
  • logic 89.0 82.0–94.0 · sd 3.5
  • perception 69.0 69.0–69.0 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

90

glm-5.3-prime standard

z-ai/glm-5.3-prime

46.6

median score

10 attempts · bubbles above, range below

Mean
47.2
SD
2.44
IQR
3.2
USD / question
0.0094
Tokens / question
1.2k
USD / 1M tok
8.04
Time / question
8.4 s
USD / test
1.24

by subject median across 10 sits

  • coding 10.0 7.0–17.2 · sd 3.0
  • data analysis 100.0 100.0–100.0 · sd 0.0
  • logic 88.0 86.0–94.0 · sd 2.6
  • perception 67.2 65.5–69.0 · sd 1.8
  • research 96.6 93.1–96.6 · sd 1.7

91

qwen3.8-27b open

qwen/qwen3.8-27b

45.8

median score

5 attempts · bubbles above, range below

Mean
46.2
SD
2.18
IQR
4.1
USD / question
0.0024
Tokens / question
1.2k
USD / 1M tok
2.01
Time / question
10.1 s
USD / test
0.322

by subject median across 5 sits

  • coding 6.8 6.0–10.9 · sd 2.1
  • data analysis 85.7 85.7–90.5 · sd 2.1
  • logic 90.0 84.0–92.0 · sd 3.3
  • perception 82.8 82.8–82.8 · sd 0.0
  • research 96.6 96.6–96.6 · sd 0.0

92

mistral-small-2603 flash

mistralai/mistral-small-2603

27.1

median score

30 attempts · bubbles above, range below

Mean
27.3
SD
0.99
IQR
1.5
USD / question
0.0001
Tokens / question
229
USD / 1M tok
0.450
Time / question
39.3 s
USD / test
0.018

by subject median across 30 sits

  • coding 16.9 12.7–22.9 · sd 2.2
  • data analysis 41.5 36.6–45.1 · sd 2.4
  • logic 25.9 21.2–31.8 · sd 2.5
  • perception 31.1 23.0–35.1 · sd 2.6
  • research 34.7 28.0–40.0 · sd 2.9

93

nova-premier-v1 frontier

amazon/nova-premier-v1

2.0

median score

30 attempts · bubbles above, range below

Mean
2.5
SD
2.30
IQR
1.2
USD / question
0.0002
Tokens / question
19
USD / 1M tok
9.37
Time / question
39.3 s
USD / test
0.033

by subject median across 30 sits

  • coding 4.9 2.2–29.9 · sd 4.9
  • data analysis 2.4 0.0–14.6 · sd 2.6
  • logic 0.0 0.0–3.5 · sd 1.0
  • perception 0.0 0.0–0.0 · sd 0.0
  • research 0.0 0.0–0.0 · sd 0.0