Latency on LLM APIs is measured along two axes — TTFT (time-to-first-token) for perceived responsiveness, and sustained tokens-per-second for output completion time. Gemini 3.1 Flash Lite Preview tops both at 321 tok/s with low TTFT on Google's infrastructure. Llama 4 Maverick comes second on 0.66s TTFT — the lowest in the catalog — and 110 tok/s. GPT-5.3-Codex (95.4 tok/s), GPT-5.4 (94.4 tok/s), and Grok 4.20 (93.8 tok/s) cluster in the 90–100 tok/s tier. Picking the right one means asking whether you optimize for the first token (chat UX) or the last token (response complete).
EDITOR'S TOP PICK
Gemini 3.
SELECTION CRITERIA
Speed isn't speed — it's two different numbers. TTFT determines whether a chat UI feels alive; sustained tok/s determines whether a long generation feels slow. We weight tok/s heaviest because most production workloads care about end-to-end completion time, but TTFT carries 30% because chat UX is felt in milliseconds.
AA's measured tokens-per-second on the model's standard endpoint. This dominates end-to-end completion time on outputs over a few hundred tokens.
View sourceThe latency between request and first streamed token. Felt directly by chat users; ≤ 1.5s feels native, ≥ 5s feels broken.
View sourceA speed ranking that ignores quality recommends models that can't answer. We exclude anything below AA Intelligence Index 18 and prefer ≥ 30.
View sourceSome providers stream in large bursts (better total time, worse perceived smoothness); others stream evenly. AA's measurements smooth this implicitly by sampling many calls.
View sourceModels that expose reasoning-effort settings (low/medium/high) let you trade quality for speed. We weight this lightly because not every workload tolerates effort-tier switching.
View sourceTOP 5 LEADERBOARD
Ranked by sustained output throughput from AA's latest snapshot, with ties broken by TTFT ascending. Quality floor: AA Intelligence Index ≥ 18 so we don't suggest a fast model that can't get the answer. All five accept OpenAI-compatible payloads on ElliotGate.
| # | Model | Provider | tokens/sec | Throughput | Price (in / out) | |
|---|---|---|---|---|---|---|
| 1 | Gemini 3.1 Flash Lite Preview | 321 tok/s | 321 tok/s | $0.25 / $1.5 | Open in ElliotGate | |
| 2 | Llama 4 Maverick | Meta | 110 tok/s | 110 tok/s | $0.15 / $0.6 | Open in ElliotGate |
| 3 | GPT-5.3-Codex | OpenAI | 95 tok/s | 95 tok/s | $1.75 / $14 | Open in ElliotGate |
| 4 | GPT-5.4 | OpenAI | 94 tok/s | 94 tok/s | $2.5 / $15 | Open in ElliotGate |
| 5 | Grok 4.20 | xAI | 94 tok/s | 94 tok/s | $2 / $6 | Open in ElliotGate |
Pricing is per 1M tokens, USD, sourced from Artificial Analysis and matched against each provider's official rate. ElliotGate charges the same per-token rate as the upstream provider.
MODEL-BY-MODEL ANALYSIS
Gemini 3.1 Flash Lite Preview leads the fastest ranking with 321 tokens per second sustained output — three times faster than the next model and roughly 5x faster than frontier reasoning models. TTFT is also in the low-latency tier (Google's infrastructure runs lean on Flash Lite). AA Intelligence Index 33.5 is the quality floor for the top slot — Flash Lite is not a frontier reasoning model, but it clears AA Intelligence Index 30 by a comfortable margin and handles routing, summarization, and structured extraction reliably. Pricing of $0.25 input / $1.5 output is the second-cheapest on this list. Best fit: real-time chat UX, streaming product surfaces, voice-driven agents where every millisecond shows.
Llama 4 Maverick records the lowest TTFT in the entire ElliotGate catalog at 0.66 seconds — meaningfully ahead of every other production model. Sustained throughput of 110 tok/s holds second on this list. The catch is the quality floor: AA Intelligence Index 18.4 is the boundary of what we accept, and Tau-2 0.178 rules it out for agent workloads. Pair Maverick with smarter slow models in a route — Maverick takes the first turn or the warm-up, smarter models handle hard branches. Best fit: streaming UX where first-token latency dominates perception (autocomplete, predictive typing, fast prefill).
GPT-5.3-Codex posts 95.4 tokens per second — the fastest OpenAI model in the speed-tier we accept and quietly the fastest model on this list with AA Intelligence Index above 50. AA Intelligence Index 53.6 keeps it in real-reasoning territory; Tau-2 0.86 supports tool use. Pricing at $1.75 input / $14 output is moderate. The trade is breadth — Codex line is specialized for software engineering, so general-knowledge tasks lean GPT-5.4 instead. For coding agents that need both speed and quality, this is often the right balance point.
GPT-5.4 posts 94.4 tok/s — within a token of GPT-5.3-Codex but with a broader knowledge profile (AA Intelligence Index 56.8 vs 53.6 for Codex). $2.5 input / $15 output sits in mid-tier pricing. The combination of frontier-tier reasoning + sub-100ms TTFT (when reasoning effort is low) + 94.4 tok/s makes it the speed champion among general-purpose frontier-quality models. Best fit: when chat UX matters but answers must hold up to scrutiny, and you don't need 5.5's marginal benchmark lead. The default choice for production assistant workloads on a latency budget.
Grok 4.20 closes the fastest top five at 93.8 tok/s with AA Intelligence Index 49.3, GPQA 0.911, and Tau-2 0.93 — strong reasoning at speed-tier latency. The headline feature is the 2M context window, twice anything else on the speed list. Pricing of $2 input / $6 output undercuts every other speed-tier model on output. The trade: AA Intelligence Index 49.3 sits below GPT-5.4 (56.8) and the Codex line. Best fit: speed-first workloads with extremely long inputs (call transcripts, codebases, document archives) where the 2M window prevents chunking overhead.
EXAMPLE PROMPTS
Paste these into the ElliotGate playground or your own SDK. Each prompt exercises a different part of the task and gives you a real signal on which model fits your workload.
Complete the next sentence for an email being composed. The user has written: 'Hi Sarah, thanks for your quick reply yesterday. I wanted to circle back on the timeline we discussed —' Continue with one sentence that sounds natural. Stop after the period.
One natural sentence streamed under 500ms. Llama 4 Maverick wins on TTFT; Gemini 3.1 Flash Lite wins on total completion time when the user keeps typing.
Explain in 250 words why JavaScript's `==` produces unexpected results for `null == undefined`, but `=== ` does not. Include the exact spec-defined behavior. Stream the answer.
Stable streaming under 3 seconds total. GPT-5.4 and GPT-5.3-Codex strike the right balance of fast streaming and accurate spec citation; pure-speed models risk surface-level explanations.
Below is a 60-page board memo (mostly tables). Produce a one-page executive summary with three sections — Decisions, Risks, Next Steps. Stream the answer. Reading begins immediately on first token; do not wait until the analysis is complete to start writing.
Substantive content arrives in the first 2 seconds. Grok 4.20's 2M context handles the long input without chunking; Gemini 3.1 Flash Lite finishes fastest overall.
QUICK START
Every ranked model accepts the same OpenAI-compatible request body. Change the model slug, keep the rest of the code, and you are routing across vendors with one API key.
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.OMINIGATE_API_KEY, // sk-omg-...
baseURL: "https://api.elliotgate.com/v1",
});
const response = await client.chat.completions.create({
model: "google/gemini-3.1-flash-lite-preview", // swap to any prod slug
messages: [{ role: "user", content: "..." }],
});QUESTIONS WE GET
Every model on this Lowest time-to-first-token and highest sustained output tokens per second ranking is one slug change away on ElliotGate. Same SDK, same balance, same dashboard.