| Model | TTFT p50 | TTFT p95 | Latency | Tok/s | Errors | N |
|---|---|---|---|---|---|---|
|
anthropic/claude-opus-4.6
|
1647ms | 3798ms | 2026ms | 55.9 | 60 | 626 |
|
anthropic/claude-opus-4.7
|
1817ms | 5285ms | 2036ms | 85.9 | 356 | 2065 |
|
anthropic/claude-opus-4.8
|
1333ms | 4091ms | 1508ms | 90.1 | 206 | 1529 |
|
anthropic/claude-sonnet-4.6
|
1865ms | 4642ms | 2361ms | 43.9 | 356 | 2065 |
|
deepseek/deepseek-v3.2
|
1296ms | 5463ms | 1889ms | 35.3 | 56 | 2365 |
|
deepseek/deepseek-v4-flash
|
523ms | 5859ms | 1980ms | 24.2 | 274 | 2147 |
|
deepseek/deepseek-v4-pro
|
0ms | 4119ms | 2202ms | 12.1 | 55 | 2366 |
|
google/gemini-2.0-flash-001
|
414ms | 1148ms | 466ms | 215.6 | 36 | 146 |
|
google/gemini-2.5-flash
|
384ms | 3266ms | 453ms | 188.6 | 266 | 2155 |
|
google/gemini-2.5-flash-lite
|
335ms | 2381ms | 399ms | 185.1 | 266 | 2155 |
|
google/gemini-3.1-flash-lite
|
597ms | 2770ms | 621ms | 534.6 | 27 | 189 |
|
google/gemini-3.5-flash
|
0ms | 844ms | 1278ms | 13.4 | 215 | 1808 |
|
openai/gpt-4o-mini
|
1122ms | 2664ms | 1249ms | 89.9 | 312 | 1807 |
|
openai/gpt-5.4
|
1072ms | 2173ms | 1318ms | 51.1 | 44 | 258 |
|
openai/gpt-5.5
|
1331ms | 3247ms | 1598ms | 54.7 | 356 | 2064 |
|
openai/gpt-oss-120b
|
0ms | 2841ms | 1390ms | 16.1 | 359 | 2062 |
|
x-ai/grok-4-fast
|
2182ms | 5544ms | 2276ms | 2034.7 | 7 | 223 |
|
x-ai/grok-4.1-fast
|
2582ms | 4196ms | 2656ms | 2659.9 | 6 | 224 |
|
x-ai/grok-4.20
|
546ms | 1441ms | 627ms | 143.1 | 58 | 2132 |
|
x-ai/grok-4.20-multi-agent
|
4024ms | 13939ms | 4384ms | 5289.2 | 1 | 143 |
|
x-ai/grok-4.3
|
1925ms | 4007ms | 2057ms | 1219.0 | 54 | 2367 |
|
x-ai/grok-build-0.1
|
1985ms | 3488ms | 2147ms | 1358.9 | 48 | 1998 |