| Model | TTFT p50 | TTFT p95 | Latency | Tok/s | Errors | N |
|---|---|---|---|---|---|---|
|
anthropic/claude-opus-4.6
|
1647ms | 3798ms | 2026ms | 55.9 | 60 | 626 |
|
anthropic/claude-opus-4.7
|
1712ms | 6056ms | 1921ms | 88.8 | 642 | 2744 |
|
anthropic/claude-opus-4.8
|
1253ms | 4272ms | 1423ms | 91.8 | 421 | 2279 |
|
anthropic/claude-sonnet-4.6
|
1720ms | 5200ms | 2186ms | 44.3 | 642 | 2744 |
|
deepseek/deepseek-v3.2
|
1322ms | 5317ms | 1891ms | 36.3 | 58 | 3328 |
|
deepseek/deepseek-v4-flash
|
412ms | 5542ms | 1908ms | 20.8 | 489 | 2897 |
|
deepseek/deepseek-v4-pro
|
0ms | 3855ms | 2164ms | 11.3 | 56 | 3330 |
|
google/gemini-2.0-flash-001
|
414ms | 1148ms | 466ms | 215.6 | 36 | 146 |
|
google/gemini-2.5-flash
|
390ms | 3620ms | 457ms | 180.9 | 481 | 2905 |
|
google/gemini-2.5-flash-lite
|
320ms | 2617ms | 382ms | 182.8 | 481 | 2905 |
|
google/gemini-3.1-flash-lite
|
597ms | 2770ms | 621ms | 534.6 | 27 | 189 |
|
google/gemini-3.5-flash
|
0ms | 768ms | 1226ms | 14.0 | 430 | 2558 |
|
openai/gpt-4o-mini
|
1138ms | 3306ms | 1258ms | 91.6 | 598 | 2486 |
|
openai/gpt-5.4
|
1072ms | 2173ms | 1318ms | 51.1 | 44 | 258 |
|
openai/gpt-5.5
|
1320ms | 3572ms | 1565ms | 58.1 | 642 | 2743 |
|
openai/gpt-oss-120b
|
0ms | 2621ms | 1372ms | 16.1 | 645 | 2741 |
|
x-ai/grok-4-fast
|
2182ms | 5544ms | 2276ms | 2034.7 | 7 | 223 |
|
x-ai/grok-4.1-fast
|
2582ms | 4196ms | 2656ms | 2659.9 | 6 | 224 |
|
x-ai/grok-4.20
|
576ms | 1441ms | 675ms | 116.4 | 59 | 3096 |
|
x-ai/grok-4.20-multi-agent
|
4024ms | 13939ms | 4384ms | 5289.2 | 1 | 143 |
|
x-ai/grok-4.3
|
1957ms | 3732ms | 2078ms | 1197.4 | 55 | 3331 |
|
x-ai/grok-build-0.1
|
2202ms | 4513ms | 2388ms | 1184.1 | 49 | 2962 |