| Model | TTFT p50 | TTFT p95 | Latency | Tok/s | Errors | N |
|---|---|---|---|---|---|---|
|
anthropic/claude-opus-4.6
|
1647ms | 3798ms | 2026ms | 55.9 | 60 | 626 |
|
anthropic/claude-opus-4.7
|
1826ms | 5282ms | 2044ms | 85.6 | 315 | 1986 |
|
anthropic/claude-opus-4.8
|
1344ms | 4075ms | 1518ms | 89.2 | 178 | 1437 |
|
anthropic/claude-sonnet-4.6
|
1881ms | 4626ms | 2376ms | 43.9 | 315 | 1986 |
|
deepseek/deepseek-v3.2
|
1291ms | 5463ms | 1892ms | 35.7 | 56 | 2245 |
|
deepseek/deepseek-v4-flash
|
540ms | 5911ms | 1974ms | 25.2 | 246 | 2055 |
|
deepseek/deepseek-v4-pro
|
0ms | 4149ms | 2196ms | 12.2 | 55 | 2246 |
|
google/gemini-2.0-flash-001
|
414ms | 1148ms | 466ms | 215.6 | 36 | 146 |
|
google/gemini-2.5-flash
|
381ms | 2696ms | 450ms | 189.1 | 238 | 2063 |
|
google/gemini-2.5-flash-lite
|
334ms | 2170ms | 398ms | 184.3 | 238 | 2063 |
|
google/gemini-3.1-flash-lite
|
597ms | 2770ms | 621ms | 534.6 | 27 | 189 |
|
google/gemini-3.5-flash
|
0ms | 853ms | 1283ms | 13.4 | 187 | 1716 |
|
openai/gpt-4o-mini
|
1122ms | 2654ms | 1244ms | 89.4 | 271 | 1728 |
|
openai/gpt-5.4
|
1072ms | 2173ms | 1318ms | 51.1 | 44 | 258 |
|
openai/gpt-5.5
|
1332ms | 3242ms | 1604ms | 54.4 | 315 | 1985 |
|
openai/gpt-oss-120b
|
0ms | 2876ms | 1386ms | 16.1 | 318 | 1983 |
|
x-ai/grok-4-fast
|
2182ms | 5544ms | 2276ms | 2034.7 | 7 | 223 |
|
x-ai/grok-4.1-fast
|
2582ms | 4196ms | 2656ms | 2659.9 | 6 | 224 |
|
x-ai/grok-4.20
|
538ms | 1456ms | 616ms | 148.2 | 58 | 2012 |
|
x-ai/grok-4.20-multi-agent
|
4024ms | 13939ms | 4384ms | 5289.2 | 1 | 143 |
|
x-ai/grok-4.3
|
1913ms | 4138ms | 2044ms | 1227.7 | 54 | 2247 |
|
x-ai/grok-build-0.1
|
1969ms | 3505ms | 2131ms | 1378.4 | 48 | 1878 |