Live LLM Inference Prices — GPT-4o, Claude & Gemini Token Rates Across 22+ Providers
What is tracked in this table?
This table shows live input and output token pricing (per 1M tokens, USD) for 262+ large language models across 33+ inference API providers. Metrics include median latency (TTFT in ms), throughput (tokens/sec), context window size, vision support, function calling support, and an intelligence score derived from public benchmarks. Prices are updated daily from provider pricing pages.
Providers covered
The table covers OpenAI (GPT-4o, GPT-4o mini, o1, o3), Anthropic (Claude 3.5 Sonnet, Claude 3 Haiku), Google (Gemini 1.5 Pro, Gemini 1.5 Flash, Gemini 2.0), Groq (Llama 3.1, Mixtral), Together AI, Fireworks AI, Mistral AI, Cohere, DeepSeek, Perplexity, Amazon Bedrock, Azure OpenAI, and more. Use the tier filter to compare frontier, advanced, efficient, and budget model tiers.
Input vs output token pricing
Most LLM APIs charge separately for input (prompt) tokens and output (completion) tokens. Output tokens are typically 3–5× more expensive than input tokens because generation is compute-intensive. For RAG and document-processing workloads with long prompts and short outputs, optimise for input price. For chatbots and code generation with long outputs, output price matters more. The table shows both rates so you can calculate true cost for your specific input/output ratio.
LLM pricing in 2026
LLM inference costs have fallen dramatically since 2023. GPT-4o is now $2.50/1M input tokens — a 92% reduction from the original GPT-4 at $30/1M. Claude 3.5 Haiku costs $0.80/1M input tokens. Open-weight models on inference APIs are even cheaper: Llama 3.1 8B is available from $0.06/1M tokens on Groq and Together AI. DeepSeek V3 offers GPT-4-class reasoning at $0.27/1M input tokens, making it one of the best value frontier models available via API.
Latency and throughput explained
Latency (TTFT — time to first token) measures how quickly the API starts responding after a request is sent. Lower is better for interactive applications. Throughput (tokens/sec) measures how fast the model generates output tokens. Groq's LPU hardware delivers the highest throughput — often 500–800 tokens/sec for Llama 3.1 70B — making it the fastest option for latency-sensitive applications. OpenAI and Anthropic prioritise reliability and model quality over raw speed.
Prompt caching and batch API discounts
Two pricing mechanisms can dramatically reduce your effective LLM API cost. Prompt caching (available on OpenAI and Anthropic) stores repeated context — system prompts, few-shot examples, RAG retrieved chunks — and charges 50–90% less for cached tokens on subsequent requests. For applications with long, repeated system prompts, caching can cut input costs by 70%+. Batch APIs (OpenAI Batch, Anthropic Message Batches) offer 50% discounts for non-real-time workloads with up to 24-hour completion windows. At 10M tokens/day, the batch discount saves $450–$1,500/month depending on model tier. Both mechanisms are reflected in the effective cost column where providers support them.
Context window size and cost implications
Context window size determines how much text a model can process in a single request. Larger windows enable full-document analysis, long conversation history, and large codebase understanding — but every token in the context window is billed as an input token. A 100,000-token context at GPT-4o pricing costs $0.25 per request. At 1,000 requests/day, that's $2,500/month from context alone. RAG (retrieval-augmented generation) reduces this by retrieving only the 2,000–5,000 most relevant tokens rather than sending the full document. Gemini 1.5 Pro's 1M-token context window is the exception — at $1.25/1M input tokens, it makes full-document analysis economically viable for the first time.