Rent H100 40GB
Compare live on-demand and spot rental prices across 97+ cloud providers. PCIe variant of H100. Lower TDP and cost than SXM5. Good for inference-heavy workloads.
Choosing the right billing model for H100 40GB
Provision and terminate at any time. Ideal for development, short experiments, and workloads with unpredictable duration.
Instances can be reclaimed when demand spikes. Best for fault-tolerant batch jobs, training with checkpointing, and preprocessing.
Lock in a rate for 1–3 months. Right for sustained production inference or long training runs where cost predictability matters.
Best use cases
- Mid-size LLM inference
- Cost-efficient training
- PCIe deployments
H100 40GB — Specs & Benchmarks
Performance bars, compute tiers (FP32/FP16/BF16/FP8/INT8), memory specs, LLM model size guidance, and related GPU comparisons.
H100 40GB Rental Guide
Rent an H100 40GB when Hopper’s FP8 path, TensorRT optimizations, or fast PCIe deployment matters more than maximum memory capacity. It is often easier to place in conventional servers than SXM hardware, but it should be benchmarked against lower-cost A100s for inference-only work that does not benefit from FP8.
Spot is a sensible option for checkpointed training and elastic batch inference, especially when the workload can restart cleanly. Choose on-demand or reserved capacity for steady serving and development teams that need dependable access to this less common 40GB Hopper configuration.
The main comparison is H100 80GB or H200 when model weights, KV cache, or batch size approach 40GB; more compute cannot rescue an out-of-memory request. A100 40GB is cheaper and has similar capacity, but it lacks Hopper FP8 support and has far lower transformer throughput.
Frequently Asked Questions
How much does it cost to rent a H100 40GB?
H100 40GB on-demand rental prices vary by provider and region. On-demand rates typically range based on availability and provider margins — use the comparison table above to see current live rates across all providers. Spot instances are generally 40–70% cheaper than on-demand but can be interrupted. Monthly cost estimates (hourly rate × 730 hours) are shown in the table for sustained workloads.
Which cloud provider has the cheapest H100 40GB?
The cheapest H100 40GB provider changes as providers update their pricing. The comparison table above shows live rates sorted by price, so the cheapest option is always at the top. Factors beyond headline price include region (latency to your users), availability (high/medium/low), and billing granularity (per-second vs per-hour minimums).
What can I run on a H100 40GB?
With 40GB of HBM3, the H100 40GB can run LLM models up to approximately 20B parameters at FP16, 40B at INT8, or 80B at INT4/GGUF quantization. Common workloads include: Mid-size LLM inference, Cost-efficient training, PCIe deployments. PCIe variant of H100. Lower TDP and cost than SXM5. Good for inference-heavy workloads.
Should I use on-demand or spot pricing for H100 40GB?
Spot instances save 40–70% vs on-demand but can be interrupted when the provider needs capacity back. Use spot for: batch inference jobs, training runs with checkpointing, preprocessing pipelines, and any workload that can tolerate interruption and restart. Use on-demand for: production inference serving, interactive workloads, and jobs that cannot be interrupted. Most providers bill per second, so short on-demand jobs are not penalized by hourly minimums.
How does the H100 40GB compare to the H100 for cloud rental?
The H100 80GB delivers 1,979 TFLOPS FP16 with 3,350 GB/s HBM3 bandwidth, compared to the H100 40GB's 1513 TFLOPS FP16 and 2000 GB/s bandwidth. The H100 is significantly more expensive — typically $2.50–$5.00/hr vs lower rates for the H100 40GB. For workloads that fit within 40GB and don't require FP8 precision, the H100 40GB often delivers better cost-per-token than the H100.
What is the memory bandwidth of the H100 40GB and why does it matter?
The H100 40GB has 2000 GB/s of memory bandwidth. For LLM inference, memory bandwidth is often more important than raw TFLOPS — each autoregressive token generation reads the full model weight matrix from VRAM, so bandwidth directly determines tokens-per-second throughput. Higher bandwidth means faster inference for the same model at the same batch size. For batch inference (processing many requests simultaneously), compute throughput becomes more important.
Can I use the H100 40GB for Stable Diffusion or image generation?
Yes — the H100 40GB is well-suited for Stable Diffusion and image generation workloads. Image generation is primarily FP32 and FP16 compute-bound, and the H100 40GB's 51 TFLOPS FP32 throughput determines images-per-second. The 40GB VRAM fits SDXL (requires ~6GB) and most ControlNet pipelines. For high-throughput image generation at scale, compare cost-per-image across providers using the GPU cost calculator.