Multimodal breadth is binary on most APIs — either the model accepts video natively or it doesn't, and the gap is steep. Gemini 3.1 Pro Preview is the only model in the catalog accepting text, image, audio, video, and file in one call, and it pairs that breadth with AA Intelligence Index 57.2. Gemini 3.1 Flash Lite covers the same modality matrix at 5x lower price and 2.2x the throughput. GPT-5.5 and GPT-5.4 anchor the text-+-image-+-file tier with frontier reasoning; Qwen3.6-Plus extends that with native video input at the cost-effective end. Real multimodal pipelines route by modality complexity, not by a single 'multimodal model' choice.
EDITOR'S TOP PICK
Gemini 3.
SELECTION CRITERIA
We separate multimodal models by which modalities they accept natively (no pre-processing), how the price scales when those modalities are sent, and whether the reasoning quality holds up after multimodal grounding. The criteria below weight modality breadth hardest, then reasoning floor, then cost on image-token pricing.
Which of text / image / audio / video / file the model accepts in one API call without pre-processing. Each additional modality that doesn't require a pipeline change is worth significant points.
View sourceA multimodal model that can see an image but can't reason about it isn't useful. We require AA Intelligence Index ≥ 30 and prefer models above 50.
View sourceVendors charge differently for visual tokens — some flat-rate per image, some per pixel-block. Cheap text rates can mask expensive image rates and vice versa.
View sourceModern multimodal agents combine 'look at this image' with 'call this tool with these args'. Tau-2 on the multimodal model determines whether tool args stay correct after visual grounding.
View sourceMultimodal payloads inflate input length significantly, which usually slows TTFT. AA-measured throughput on the unimodal text baseline is a reasonable proxy ranked descending.
View sourceTOP 5 LEADERBOARD
Ranked by modality coverage breadth, with ties broken by AA Intelligence Index. Each of the top five accepts at least three input modalities. All are reachable through one ElliotGate API key — multimodal payloads use the same OpenAI-compatible parts array as text-only payloads.
| # | Model | Provider | Modality breadth | Throughput | Price (in / out) | |
|---|---|---|---|---|---|---|
| 1 | Gemini 3.1 Pro Preview | 5 input modalities | 143 tok/s | $2 / $12 | Open in ElliotGate | |
| 2 | Gemini 3.1 Flash Lite Preview | 5 input modalities | 321 tok/s | $0.25 / $1.5 | Open in ElliotGate | |
| 3 | GPT-5.5 | OpenAI | 3 input modalities | 64 tok/s | $5 / $30 | Open in ElliotGate |
| 4 | Qwen3.6 Plus | Qwen | 3 input modalities | 52 tok/s | $0.5 / $3 | Open in ElliotGate |
| 5 | GPT-5.4 | OpenAI | 3 input modalities | 94 tok/s | $2.5 / $15 | Open in ElliotGate |
Pricing is per 1M tokens, USD, sourced from Artificial Analysis and matched against each provider's official rate. ElliotGate charges the same per-token rate as the upstream provider.
MODEL-BY-MODEL ANALYSIS
Gemini 3.1 Pro Preview is the only model in the ElliotGate catalog accepting text, image, audio, video, and file in one API call. AA Intelligence Index 57.2 keeps it at frontier reasoning quality, GPQA Diamond 0.941 is the highest in the multimodal ranking, and throughput at 142.7 tok/s prevents multimodal input length from blocking the response. Pricing at $2 input / $12 output is mid-tier — image-token amortization is included rather than charged as a separate rate. Best fit: multimodal RAG over video archives, accessibility tools that combine speech transcription with vision, document-and-recording workflows where one model handles everything.
Gemini 3.1 Flash Lite Preview matches Gemini Pro on modality breadth — text, image, audio, video, file — and is the throughput champion of the entire ElliotGate catalog at 321 tokens per second. AA Intelligence Index 33.5 limits its reasoning ceiling, but for triage-style multimodal tasks (which video clip contains an outage, which document references the contract clause) the ceiling rarely binds. Pricing of $0.25 input / $1.5 output is 8x lower than Gemini Pro on input. Best fit: high-volume multimodal triage, accessibility captions, real-time multimodal moderation.
GPT-5.5 accepts text, image, and file but does not natively accept audio or video — those have to be pre-processed (Whisper for audio, frame-extraction for video). For workloads where the visual modality is image-only (document understanding, OCR-driven extraction, screenshot analysis), 5.5's frontier reasoning makes it the best multimodal pick on raw quality: AA Intelligence Index 60.2, GPQA 0.935. Pricing of $5 input / $30 output is the multimodal ranking's highest, and image tokens compound that. Best fit: high-value-per-call document-understanding workflows where the answer accuracy is worth the multimodal premium.
Qwen3.6-Plus accepts text, image, and video natively — file and audio are not in its modality matrix. AA Intelligence Index 50 sits mid-tier, but Tau-2 of 0.977 (the single highest on our ranking) means tool calls after visual grounding stay reliable. Pricing of $0.5 input / $3 output makes it the cost leader among multimodal-with-video options. Best fit: video-aware agent workloads at scale (content moderation, video-driven dashboards, video-to-action automation) where Gemini's preview status is a liability and a stable vendor relationship matters.
GPT-5.4 closes the multimodal top five with the same modality matrix as 5.5 (text + image + file) at half the price: $2.5 input / $15 output. AA Intelligence Index 56.8 is essentially indistinguishable from 5.5's at 60.2 for everyday vision tasks (OCR, screenshot QA, document layout). For document-understanding workloads that don't need 5.5's marginal benchmark lead, 5.4 is the obvious default. Throughput at 94.4 tok/s is also faster than 5.5 on the multimodal baseline. Best fit: most production multimodal applications that center on documents and screenshots rather than video.
EXAMPLE PROMPTS
Paste these into the ElliotGate playground or your own SDK. Each prompt exercises a different part of the task and gives you a real signal on which model fits your workload.
Below is a 14-page scanned contract as a PDF attachment. Find every signature block and report: (a) party name, (b) page number, (c) whether the signature line is filled. Return JSON only. Then on a separate line, flag any signature block that appears unfilled.
Clean JSON, no prose. GPT-5.5 wins on document-layout reasoning; Gemini 3.1 Pro is the fastest. GPT-5.4 is the cost-quality middle.
Below is a 90-second video of a manufacturing line. Identify every (a) machine state change, (b) operator action, with timestamps in mm:ss. Return CSV (event_type,timestamp,description). Do not narrate; just emit rows.
Only Gemini Pro / Flash Lite and Qwen3.6-Plus accept video natively; the OpenAI models in this ranking would fail without frame extraction upstream.
I will send three artifacts: an alert screenshot, a 30-second audio voice memo from on-call, and the relevant log file. Classify the incident severity (P0/P1/P2/P3), name the most likely root cause, and propose the next three actions. Return exactly one JSON object.
A canonical multimodal-triage workload — Gemini Pro is the only model that can read all three modalities in one call. GPT-5.5 would need Whisper pre-processing on the audio.
QUICK START
Every ranked model accepts the same OpenAI-compatible request body. Change the model slug, keep the rest of the code, and you are routing across vendors with one API key.
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.OMINIGATE_API_KEY, // sk-omg-...
baseURL: "https://api.elliotgate.com/v1",
});
const response = await client.chat.completions.create({
model: "google/gemini-3.1-pro-preview", // swap to any prod slug
messages: [{ role: "user", content: "..." }],
});QUESTIONS WE GET
Every model on this Combined text + image + audio + video input on a single API ranking is one slug change away on ElliotGate. Same SDK, same balance, same dashboard.