Live LLM Inference Prices — GPT-4o, Claude & Gemini Token Rates
What is tracked in this table?
This table shows live input and output token pricing (per 1M tokens, USD) for large language models across tracked inference API providers. Metrics include median latency (TTFT in ms), throughput (tokens/sec), context window size, vision support, function calling support, and an intelligence score derived from public benchmarks. Prices are updated every 15 minutes from provider pricing pages.
Providers covered
The table covers OpenAI (GPT-4o, GPT-4o mini, o1, o3), Anthropic (Claude 3.5 Sonnet, Claude 3 Haiku), Google (Gemini 1.5 Pro, Gemini 1.5 Flash, Gemini 2.0), Groq (Llama 3.1, Mixtral), Together AI, Fireworks AI, Mistral AI, Cohere, DeepSeek, Perplexity, Amazon Bedrock, Azure OpenAI, and more. Use the tier filter to compare frontier, advanced, efficient, and budget model tiers.
Input vs output token pricing
Most LLM APIs charge separately for input (prompt) tokens and output (completion) tokens. Output tokens are typically 3–5× more expensive than input tokens because generation is compute-intensive. For RAG and document-processing workloads with long prompts and short outputs, optimise for input price. For chatbots and code generation with long outputs, output price matters more. The table shows both rates so you can calculate true cost for your specific input/output ratio.
LLM pricing trends
LLM inference costs have fallen dramatically since 2023. Frontier model prices have dropped by 80–95% as providers compete and hardware efficiency improves. Open-weight models on inference APIs are significantly cheaper than proprietary frontier models. Check the live table above for current rates — prices change frequently as providers update their offerings.
Latency and throughput explained
Latency (TTFT — time to first token) measures how quickly the API starts responding after a request is sent. Lower is better for interactive applications. Throughput (tokens/sec) measures how fast the model generates output tokens. Groq's LPU hardware delivers high throughput for Llama models, making it a strong option for latency-sensitive applications. OpenAI and Anthropic prioritise reliability and model quality alongside speed.
Prompt caching and batch API discounts
Two pricing mechanisms can dramatically reduce your effective LLM API cost. Prompt caching (available on OpenAI and Anthropic) stores repeated context — system prompts, few-shot examples, RAG retrieved chunks — and charges 50–90% less for cached tokens on subsequent requests. For applications with long, repeated system prompts, caching can cut input costs by 70%+. Batch APIs (OpenAI Batch, Anthropic Message Batches) offer 50% discounts for non-real-time workloads with up to 24-hour completion windows. Both mechanisms are reflected in the effective cost column where providers support them.
Context window size and cost implications
Context window size determines how much text a model can process in a single request. Larger windows enable full-document analysis, long conversation history, and large codebase understanding — but every token in the context window is billed as an input token. RAG (retrieval-augmented generation) reduces this by retrieving only the most relevant tokens rather than sending the full document. Check the live table for current context window sizes and pricing across providers.
Get token price alerts
Enter your email and a price threshold. We'll notify you when input token pricing drops to your target.