| Model | TTFT p50 | TTFT p95 | Latency | Tok/s | Errors | N |
|---|---|---|---|---|---|---|
|
anthropic/claude-opus-4.6
|
1647ms | 3798ms | 2026ms | 55.9 | 60 | 626 |
|
anthropic/claude-opus-4.7
|
1760ms | 6034ms | 1971ms | 87.9 | 570 | 2432 |
|
anthropic/claude-opus-4.8
|
1289ms | 4444ms | 1457ms | 91.7 | 367 | 1949 |
|
anthropic/claude-sonnet-4.6
|
1772ms | 5184ms | 2273ms | 44.2 | 570 | 2432 |
|
deepseek/deepseek-v3.2
|
1324ms | 5317ms | 1896ms | 35.5 | 56 | 2946 |
|
deepseek/deepseek-v4-flash
|
399ms | 5799ms | 1974ms | 20.7 | 435 | 2567 |
|
deepseek/deepseek-v4-pro
|
0ms | 3973ms | 2201ms | 11.7 | 55 | 2947 |
|
google/gemini-2.0-flash-001
|
414ms | 1148ms | 466ms | 215.6 | 36 | 146 |
|
google/gemini-2.5-flash
|
385ms | 5451ms | 453ms | 186.5 | 427 | 2575 |
|
google/gemini-2.5-flash-lite
|
330ms | 2990ms | 392ms | 186.3 | 427 | 2575 |
|
google/gemini-3.1-flash-lite
|
597ms | 2770ms | 621ms | 534.6 | 27 | 189 |
|
google/gemini-3.5-flash
|
0ms | 792ms | 1255ms | 13.6 | 376 | 2228 |
|
openai/gpt-4o-mini
|
1136ms | 3141ms | 1257ms | 91.2 | 526 | 2174 |
|
openai/gpt-5.4
|
1072ms | 2173ms | 1318ms | 51.1 | 44 | 258 |
|
openai/gpt-5.5
|
1317ms | 3525ms | 1574ms | 56.4 | 570 | 2431 |
|
openai/gpt-oss-120b
|
0ms | 2780ms | 1391ms | 16.0 | 573 | 2429 |
|
x-ai/grok-4-fast
|
2182ms | 5544ms | 2276ms | 2034.7 | 7 | 223 |
|
x-ai/grok-4.1-fast
|
2582ms | 4196ms | 2656ms | 2659.9 | 6 | 224 |
|
x-ai/grok-4.20
|
566ms | 1399ms | 666ms | 120.0 | 58 | 2713 |
|
x-ai/grok-4.20-multi-agent
|
4024ms | 13939ms | 4384ms | 5289.2 | 1 | 143 |
|
x-ai/grok-4.3
|
1938ms | 3740ms | 2063ms | 1204.9 | 54 | 2948 |
|
x-ai/grok-build-0.1
|
2112ms | 4185ms | 2299ms | 1234.2 | 48 | 2579 |