| Model | TTFT p50 | TTFT p95 | Latency | Tok/s | Errors | N |
|---|---|---|---|---|---|---|
|
anthropic/claude-haiku-5.5
|
874ms | 5965ms | 1068ms | 64.1 | 178 | 488 |
|
anthropic/claude-opus-4.6
|
1647ms | 3798ms | 2026ms | 55.9 | 60 | 626 |
|
anthropic/claude-opus-4.7
|
1693ms | 6071ms | 1888ms | 89.7 | 719 | 2975 |
|
anthropic/claude-opus-4.8
|
1234ms | 4300ms | 1395ms | 92.3 | 471 | 2537 |
|
anthropic/claude-opus-5
|
0ms | 0ms | 2876ms | 7.0 | 24 | 72 |
|
anthropic/claude-opus-5.5
|
1708ms | 6412ms | 2389ms | 22.8 | 240 | 538 |
|
anthropic/claude-sonnet-4.6
|
1674ms | 5242ms | 2136ms | 44.3 | 719 | 2975 |
|
anthropic/claude-sonnet-5
|
1280ms | 2288ms | 1803ms | 30.0 | 4 | 12 |
|
anthropic/claude-sonnet-5.5
|
1113ms | 6052ms | 1472ms | 41.0 | 213 | 565 |
|
deepseek/deepseek-v3.2
|
1332ms | 5623ms | 1900ms | 37.1 | 63 | 3631 |
|
deepseek/deepseek-v4-flash
|
0ms | 5112ms | 1888ms | 18.5 | 539 | 3434 |
|
deepseek/deepseek-v4-flash-0731
|
0ms | 1369ms | 1036ms | 22.5 | 10 | 768 |
|
deepseek/deepseek-v4-pro
|
0ms | 3692ms | 2131ms | 11.1 | 57 | 3637 |
|
deepseek/deepseek-v4.1-flash
|
0ms | 1001ms | 895ms | 26.8 | 117 | 661 |
|
deepseek/deepseek-v4.1-flash-1017
|
0ms | 0ms | 516ms | 39.7 | 73 | 426 |
|
google/gemini-2.0-flash-001
|
414ms | 1148ms | 466ms | 215.6 | 36 | 146 |
|
google/gemini-2.5-flash
|
390ms | 4231ms | 458ms | 176.6 | 531 | 3163 |
|
google/gemini-2.5-flash-lite
|
311ms | 3822ms | 378ms | 172.7 | 664 | 3429 |
|
google/gemini-3.1-flash-lite
|
572ms | 5892ms | 629ms | 179.6 | 180 | 670 |
|
google/gemini-3.5-flash
|
0ms | 755ms | 1218ms | 14.0 | 480 | 2816 |
|
google/gemini-3.8-flash
|
0ms | 1724ms | 2027ms | 8.6 | 179 | 599 |
|
google/gemini-4-argon
|
2962ms | 12285ms | 3048ms | 141.7 | 101 | 422 |
|
openai/gpt-4o-mini
|
1136ms | 3495ms | 1256ms | 92.3 | 675 | 2717 |
|
openai/gpt-5.4
|
1072ms | 2173ms | 1318ms | 51.1 | 44 | 258 |
|
openai/gpt-5.5
|
1312ms | 3788ms | 1550ms | 60.1 | 719 | 2974 |
|
openai/gpt-5.6-luna
|
1124ms | 5970ms | 1339ms | 73.7 | 240 | 538 |
|
openai/gpt-5.6-sol
|
1313ms | 6452ms | 1453ms | 73.4 | 15 | 49 |
|
openai/gpt-6-luna
|
0ms | 3408ms | 2508ms | 5.6 | 240 | 538 |
|
openai/gpt-6.1-sol
|
1680ms | 6367ms | 1906ms | 57.0 | 225 | 489 |
|
openai/gpt-oss-120b
|
0ms | 2562ms | 1360ms | 16.1 | 722 | 2972 |
|
x-ai/grok-4-fast
|
2182ms | 5544ms | 2276ms | 2034.7 | 7 | 223 |
|
x-ai/grok-4.1-fast
|
2582ms | 4196ms | 2656ms | 2659.9 | 6 | 224 |
|
x-ai/grok-4.20
|
587ms | 1441ms | 684ms | 116.8 | 60 | 3403 |
|
x-ai/grok-4.20-multi-agent
|
4024ms | 13939ms | 4384ms | 5289.2 | 1 | 143 |
|
x-ai/grok-4.3
|
1962ms | 3640ms | 2087ms | 1188.9 | 56 | 3638 |
|
x-ai/grok-4.5
|
1130ms | 3060ms | 1273ms | 223.2 | 9 | 769 |
|
x-ai/grok-4.6
|
2508ms | 5027ms | 2535ms | 2939.1 | 11 | 767 |
|
x-ai/grok-4.7
|
1284ms | 4671ms | 1323ms | 1547.0 | 14 | 764 |
|
x-ai/grok-build-0.1
|
2242ms | 4565ms | 2430ms | 1153.3 | 51 | 3268 |