| Model | TTFT p50 | TTFT p95 | Latency | Tok/s | Errors | N |
|---|---|---|---|---|---|---|
|
anthropic/claude-opus-4.6
|
1647ms | 3798ms | 2026ms | 55.9 | 60 | 626 |
|
anthropic/claude-opus-4.7
|
1694ms | 6081ms | 1892ms | 89.6 | 718 | 2956 |
|
anthropic/claude-opus-4.8
|
1236ms | 4384ms | 1397ms | 92.2 | 470 | 2518 |
|
anthropic/claude-sonnet-4.6
|
1676ms | 5261ms | 2144ms | 44.3 | 718 | 2956 |
|
deepseek/deepseek-v3.2
|
1332ms | 5463ms | 1900ms | 37.0 | 63 | 3611 |
|
deepseek/deepseek-v4-flash
|
300ms | 5380ms | 1889ms | 19.9 | 538 | 3136 |
|
deepseek/deepseek-v4-pro
|
0ms | 3693ms | 2120ms | 11.2 | 57 | 3617 |
|
google/gemini-2.0-flash-001
|
414ms | 1148ms | 466ms | 215.6 | 36 | 146 |
|
google/gemini-2.5-flash
|
389ms | 4245ms | 456ms | 176.9 | 530 | 3144 |
|
google/gemini-2.5-flash-lite
|
314ms | 2800ms | 379ms | 177.6 | 530 | 3144 |
|
google/gemini-3.1-flash-lite
|
597ms | 2770ms | 621ms | 534.6 | 27 | 189 |
|
google/gemini-3.5-flash
|
0ms | 755ms | 1220ms | 14.0 | 479 | 2797 |
|
openai/gpt-4o-mini
|
1136ms | 3520ms | 1257ms | 91.9 | 674 | 2698 |
|
openai/gpt-5.4
|
1072ms | 2173ms | 1318ms | 51.1 | 44 | 258 |
|
openai/gpt-5.5
|
1312ms | 3864ms | 1551ms | 59.9 | 718 | 2955 |
|
openai/gpt-oss-120b
|
0ms | 2578ms | 1360ms | 16.1 | 721 | 2953 |
|
x-ai/grok-4-fast
|
2182ms | 5544ms | 2276ms | 2034.7 | 7 | 223 |
|
x-ai/grok-4.1-fast
|
2582ms | 4196ms | 2656ms | 2659.9 | 6 | 224 |
|
x-ai/grok-4.20
|
587ms | 1440ms | 684ms | 116.8 | 60 | 3383 |
|
x-ai/grok-4.20-multi-agent
|
4024ms | 13939ms | 4384ms | 5289.2 | 1 | 143 |
|
x-ai/grok-4.3
|
1962ms | 3660ms | 2087ms | 1188.9 | 56 | 3618 |
|
x-ai/grok-build-0.1
|
2238ms | 4565ms | 2426ms | 1152.8 | 51 | 3248 |