| Model | TTFT p50 | TTFT p95 | Latency | Tok/s | Errors | N |
|---|---|---|---|---|---|---|
|
anthropic/claude-opus-4.6
|
1647ms | 3798ms | 2026ms | 55.9 | 60 | 626 |
|
anthropic/claude-opus-4.7
|
1870ms | 4983ms | 2100ms | 84.7 | 179 | 1400 |
|
anthropic/claude-opus-4.8
|
1328ms | 3186ms | 1498ms | 90.1 | 87 | 806 |
|
anthropic/claude-sonnet-4.6
|
1872ms | 4243ms | 2358ms | 44.0 | 179 | 1400 |
|
deepseek/deepseek-v3.2
|
1294ms | 5040ms | 1934ms | 35.1 | 55 | 1524 |
|
deepseek/deepseek-v4-flash
|
805ms | 5799ms | 1952ms | 45.0 | 151 | 1428 |
|
deepseek/deepseek-v4-pro
|
0ms | 4171ms | 2203ms | 13.3 | 52 | 1527 |
|
google/gemini-2.0-flash-001
|
414ms | 1148ms | 466ms | 215.6 | 36 | 146 |
|
google/gemini-2.5-flash
|
392ms | 2335ms | 460ms | 194.3 | 147 | 1432 |
|
google/gemini-2.5-flash-lite
|
332ms | 1042ms | 393ms | 188.5 | 147 | 1432 |
|
google/gemini-3.1-flash-lite
|
597ms | 2770ms | 621ms | 534.6 | 27 | 189 |
|
google/gemini-3.5-flash
|
0ms | 795ms | 1269ms | 13.5 | 96 | 1085 |
|
openai/gpt-4o-mini
|
1122ms | 2543ms | 1238ms | 92.6 | 135 | 1142 |
|
openai/gpt-5.4
|
1072ms | 2173ms | 1318ms | 51.1 | 44 | 258 |
|
openai/gpt-5.5
|
1369ms | 2964ms | 1656ms | 50.8 | 179 | 1399 |
|
openai/gpt-oss-120b
|
0ms | 3056ms | 1374ms | 16.7 | 180 | 1399 |
|
x-ai/grok-4-fast
|
2182ms | 5544ms | 2276ms | 2034.7 | 7 | 223 |
|
x-ai/grok-4.1-fast
|
2582ms | 4196ms | 2656ms | 2659.9 | 6 | 224 |
|
x-ai/grok-4.20
|
523ms | 1604ms | 590ms | 157.7 | 58 | 1290 |
|
x-ai/grok-4.20-multi-agent
|
4024ms | 13939ms | 4384ms | 5289.2 | 1 | 143 |
|
x-ai/grok-4.3
|
1830ms | 4386ms | 1941ms | 1264.0 | 54 | 1525 |
|
x-ai/grok-build-0.1
|
1980ms | 3448ms | 2124ms | 1573.0 | 48 | 1156 |