SambaNova Cloud
High-throughput inference on SambaNova RDU chips — gpt-oss-120b, GLM-5.3, Llama
SambaNova Cloud runs inference on SambaNova's proprietary Reconfigurable Dataflow Unit (RDU) chips, delivering very high throughput (800+ tokens/sec on gpt-oss-120b). The catalog covers gpt-oss-120b, GLM-5.3, MiniMax-M3, DeepSeek-V3.2, Meta-Llama-3.3-70B, and Gemma 4 31B. Pricing is PAYG with prompt caching on select models.
Proprietary RDU chips deliver industry-leading throughput on open-weight models at competitive per-token rates.
- Proprietary RDU chips deliver very high throughput — 800+ tokens/sec on gpt-oss-120b
- Competitive pricing on gpt-oss-120b ($0.22/$0.59 per 1M)
- Prompt caching on GLM-5.3 (cached input at $0.14/1M)
- Strong open-weight model catalog: Llama, DeepSeek, Gemma, MiniMax
- Smaller model catalog vs OpenRouter or Together AI
- No proprietary frontier models — inference-only platform
- RDU architecture may behave differently from GPU inference for some workloads
SambaNova Cloud Models
| Model | Modality | Tier | Price | Cached /1M | TTFT | Throughput | Context | Intel. | Caps | |
|---|---|---|---|---|---|---|---|---|---|---|
gpt-oss-120b GPT-OSS | Chat | efficient | $0.22/ $0.59 in / out per 1M EST Oct 2026 | — | 80ms | 800 t/s | 131K | 74 | ||
gemma-4-31B Gemma 4 | Chat | efficient | $0.38/ $1.15 in / out per 1M EST Oct 2026 | — | 250ms | 250 t/s | 131K | 75 | ||
MiniMax-M3 MiniMax | Chat | efficient | $0.60/ $2.40 in / out per 1M EST Oct 2026 | — | 300ms | 200 t/s | 131K | 78 | ||
Meta-Llama-3.3-70B-Instruct Llama 3.3 | Chat | efficient | $0.60/ $1.20 in / out per 1M EST Oct 2026 | — | — | — | 131K | 76 | ||
GLM-5.3 GLM-5.3 | Chat | advanced | $1.40/ $4.40 in / out per 1M EST Oct 2026 | $0.14 | — | — | 131K | — | ||
DeepSeek-V3.2 DeepSeek | Chat | advanced | $3.00/ $4.50 in / out per 1M EST Oct 2026 | — | — | — | 131K | — |
SambaNova Cloud — Frequently Asked Questions (FAQs)
Similar Providers
SambaNova Cloud Reviews
All LLM Providers
Full comparison tableSambaNova Cloud LLM pricing overview
SambaNova Cloud is an inference API provider that hosts open-weight and third-party large language models. SambaNova Cloud runs inference on SambaNova's proprietary Reconfigurable Dataflow Unit (RDU) chips, delivering very high throughput (800+ tokens/sec on gpt-oss-120b). The catalog covers gpt-oss-120b, GLM-5.3, MiniMax-M3, DeepSeek-V3.2, Meta-Llama-3.3-70B, and Gemma 4 31B. Pricing is PAYG with prompt caching on select models. Common use cases include High-throughput inference, Cost-sensitive workloads, Open-weight models. Headquartered in Palo Alto, CA, founded 2017. All models are accessible via a REST API compatible with standard OpenAI-style request formats, enabling drop-in integration with most LLM frameworks and orchestration tools.
SambaNova Cloud vs other LLM providers
SambaNova Cloud competes with OpenAI, Anthropic, Google, Groq, Together AI, Mistral, Cohere, and other inference API providers across dimensions of price, throughput, latency, context window, and model intelligence. Proprietary RDU chips deliver industry-leading throughput on open-weight models at competitive per-token rates. Use the LLM provider comparison tool to see SambaNova Cloud token pricing, latency, and throughput side-by-side with any other provider. The full LLM pricing table shows all providers ranked by input token cost, output token cost, and throughput in a single sortable view.
Why choose SambaNova Cloud?
SambaNova Cloud stands out because: Proprietary RDU chips deliver industry-leading throughput on open-weight models at competitive per-token rates.. The API currently offers 6 models, spanning 2 advanced, 4 efficient tiers. For teams running high-volume inference workloads, prompt caching and batch API endpoints can reduce effective input token costs by 50–90% — check the context window cost guide for a full breakdown of caching economics.
Understanding SambaNova Cloud token pricing
SambaNova Cloud charges separately for input (prompt) and output (completion) tokens, priced per 1M tokens in USD. Output tokens are typically 3–5× more expensive than input tokens due to the compute cost of autoregressive generation. The cheapest {meta.name} input rate is currently $0.22/1M tokens. Prices shown are sourced from SambaNova Cloud's public pricing page and updated every 15 minutes. Need help estimating your spend? Read the LLM API cost calculator guide — it covers tokens, context windows, prompt caching, and batch discounts with worked examples for RAG, chat history, and document processing workloads.
SambaNova Cloud context window and model capabilities
Context window size directly affects both capability and cost — every token in the context window is charged as an input token. SambaNova Cloud's largest context window is 131K tokens. For RAG and document processing workloads, longer context windows enable richer retrieval but increase per-call costs proportionally. Prompt caching — where supported — stores the KV state of repeated prefixes and charges 75–90% less for cache hits, making it the most impactful cost optimization for applications with consistent system prompts or retrieved documents. See the LLM context window cost guide for a full analysis of how context length affects your API bill.
Self-hosted vs managed inference: when SambaNova Cloud makes sense
Managed inference APIs like SambaNova Cloud eliminate infrastructure overhead — no GPU provisioning, driver management, or model serving stack to maintain. The trade-off is cost at scale: a single H100 at ~$2.50/hr can serve ~500K tokens/min of Llama 3.3 70B, which at SambaNova Cloud API rates would cost significantly more per token. The break-even point depends on your request volume, latency requirements, and engineering capacity. For teams processing fewer than ~10M tokens/day, managed APIs are almost always cheaper when total cost of ownership is considered. Above that threshold, self-hosted inference on rented GPU compute typically wins on unit economics. Read the cheapest GPU cloud guide for a full break-even analysis. Historical SambaNova Cloud token price data is available in the LLM price history charts.
Compare SambaNova Cloud With Another LLM Provider
Compare SambaNova Cloud vs other providers
Side-by-side token pricing, latency, throughput, and context window. View all LLM provider comparisons →