GPU Providers Compared

The Short Answer

Skip to the bottom if you want the full breakdown. But if you need a decision right now:

• Training large models (70B+): CoreWeave. Best H100 cluster networking at non-hyperscaler prices.
• Inference at scale with SLAs: AWS or GCP. You’re paying for reliability and managed tooling, not just GPUs.
• Budget-conscious teams: RunPod or Vast.ai for spot-tolerant workloads. Lambda Labs for reserved on-demand.
• Quick experiments and prototyping: Paperspace or RunPod. Fastest time from zero to interactive GPU session.
• Together AI: Not a raw GPU provider — but if you want managed inference on open models without running your own cluster, it belongs in this conversation.

Now let’s justify those calls.

What You’re Actually Comparing

GPU providers vary across six dimensions that matter for real workloads:

1. GPU selection — Which chips are available and in what configurations
2. Pricing — On-demand vs. reserved vs. spot, and whether the number is honest
3. Availability — Whether you can actually get what you need when you need it
4. Network and storage — Critical for distributed training; often ignored until it breaks
5. Ease of setup — Time from account creation to first training job
6. Support and reliability — SLAs, incident response, and whether the hardware actually works

The hyperscalers (AWS, GCP, Azure) and the GPU-native clouds (CoreWeave, Lambda, RunPod) serve fundamentally different use cases. Conflating them leads to bad purchasing decisions.

Hyperscaler GPU Clouds

AWS

AWS offers the widest range of GPU instance types but the ecosystem complexity is real — you’re navigating EC2, SageMaker, EKS, and a web of supporting services before your first job runs.

Available GPU instances:

Pricing (on-demand, approximate):

• p4d.24xlarge (8x A100 40GB): ~$32/hr
• p5.48xlarge (8x H100): ~$98/hr
• g5.xlarge (1x A10G): ~$1/hr

Spot pricing cuts these by 40–70% but H100 spot availability is unreliable.

Network: EFA (Elastic Fabric Adapter) on p4d and p5 gives you 400–3,200 Gbps bisectional bandwidth within a placement group. This matters for multi-node training and AWS does it properly. FSx for Lustre is the standard high-throughput storage option, but setup is non-trivial.

Verdict: AWS is the right choice when your organization is already AWS-native and needs to co-locate GPU compute with existing data pipelines, compliance boundaries, or managed ML services (SageMaker, Bedrock). You’re paying a 30–50% premium over GPU-native clouds for that integration convenience.

GCP

Google’s GPU lineup is competitive at the top end, and TPUs remain a legitimate alternative for transformer workloads at scale if you’re willing to invest in XLA-compatible code.

Available GPU instances:

Pricing (on-demand, approximate):

• a2-highgpu-8g (8x A100 40GB): ~$26/hr
• a3-highgpu-8g (8x H100): ~$98/hr
• g2-standard-4 (1x L4): ~$0.70/hr

GCP’s committed use discounts (1-year, 3-year) are more flexible than AWS Reserved Instances and often worth modeling for steady-state inference clusters.

Network: The a3-megagpu variant uses NVIDIA Quantum-2 InfiniBand, which is directly comparable to what CoreWeave offers. For serious multi-node training, this matters. Standard a3-highgpu uses Google’s own high-bandwidth network fabric — still fast, but not InfiniBand.

Verdict: GCP is the strongest hyperscaler option for multi-node H100 training, especially if you’re already using Google’s data services (BigQuery, GCS). The a3-mega tier is genuinely competitive with GPU-native clouds on networking. TPUs are worth evaluating if you’re training transformers from scratch at scale.

Azure

Azure’s GPU story is focused on large enterprise accounts and HPC clusters. The NDv4 and NDv5 series are their serious training hardware, but availability outside of select regions is spotty.

Available GPU instances:

Pricing: Similar to AWS at the high end. NDm A100 v4 (8x A100) runs ~$30/hr. H100 configurations are in the $90+/hr range.

Network: NDv4 and NDv5 use InfiniBand (HDR and NDR respectively), which is a real differentiator for distributed training. Azure’s HPC-optimized instances are legitimately well-networked.

Verdict: Azure is the right answer if you’re inside a Microsoft Enterprise Agreement or your organization already leans Azure. The HPC networking is solid. The self-service experience for GPU provisioning is rougher than AWS or GCP, and availability for H100s outside of East US / West Europe is genuinely constrained.

GPU-Native Clouds

CoreWeave

CoreWeave is the provider that large AI labs use when they need GPU clusters that aren’t labeled “hyperscaler tax.” They started as a crypto mining operation and pivoted to GPU compute — which means their physical infrastructure is purpose-built for GPU density, not retrofitted from a general-purpose cloud.

Available hardware:

• H100 80GB SXM (InfiniBand HDR / NDR networking)
• A100 80GB SXM / PCIe
• A40, RTX A6000 (for inference)

Pricing:

• H100 SXM (reserved): ~$2.47/hr per GPU; on-demand: ~$6.16/hr per GPU
• A100 80GB: ~$2.06/hr per GPU
• A100 40GB: ~$1.79/hr per GPU

On committed pricing these are significantly below hyperscaler on-demand rates for equivalent hardware. Compare: 8x H100 on CoreWeave reserved at ~$19.76/hr vs. ~$55–$98/hr on AWS p5.48xlarge on-demand.

Network: This is where CoreWeave wins. Their H100 cluster networking uses NVIDIA Quantum-2 InfiniBand at 400 Gbps per GPU. For multi-node training jobs that are collective-communication-bound (which large model training almost always is), this is the differentiator. They also offer Weka distributed filesystem and NFS mounts — proper HPC storage, not bolted-on blob storage.

Availability: Constrained on the best hardware. You typically need to reserve capacity in advance. They’re not a click-and-get-H100s situation for large clusters.

Verdict: CoreWeave is the right choice for teams that need dedicated multi-node H100 clusters at below-hyperscaler pricing. The networking is top-tier. The tradeoff is less managed-service ecosystem and more direct infrastructure management.

Lambda Labs

Lambda Labs built a developer-friendly GPU cloud specifically for ML engineers. Their interface is clean, pricing is predictable, and they offer both on-demand and reserved instances.

Available hardware:

• H100 SXM (1x, 8x cluster configurations)
• A100 40GB (8x cluster)
• A10 (1x–8x)

Pricing:

• H100 SXM: ~$3.99–$4.29/hr per GPU on-demand (8x clusters at lower end)
• A100 SXM 40GB: ~$1.99/hr per GPU on-demand
• A10: ~$1.29/hr per GPU on-demand

Availability: Better than CoreWeave for single-node on-demand. Eight-node H100 clusters require advance booking. They’re honest about availability on their website, which is appreciated.

Network: H100 cluster nodes are interconnected with InfiniBand. A100 cluster nodes use 200 Gbps Ethernet. Storage is NFS-based — fine for most workloads, not HPC-grade.

Ease of setup: Excellent. SSH access, persistent storage volumes, Jupyter support. You can be running a job within 15 minutes of account creation. Their filesystem is shared across instances in a cluster, which simplifies multi-node setup considerably.

Verdict: Lambda Labs is the best on-demand GPU cloud for teams that want GPU-native pricing without infrastructure engineering overhead. Ideal for single-node training, fine-tuning, and teams that run frequent experiments without long reservation commitments.

RunPod

RunPod is a marketplace model — you’re renting GPU capacity from a mix of data centers and, for community cloud, individual operators. This drives pricing down significantly but means reliability varies.

Available hardware: Highly varied. H100, A100, A40, A10G, 4090, 3090, and more — depending on availability.

Pricing:

• H100 SXM: ~$2.29–$2.79/hr on secure cloud; $1.69–$2.19 on community cloud
• A100 80GB: ~$1.64–$1.99/hr
• A10G: ~$0.44–$0.64/hr
• RTX 4090: ~$0.34–$0.44/hr

Availability: Generally good for consumer-grade GPUs (4090s) and A-series. H100 availability fluctuates. Community cloud is cheaper but hardware quality varies.

Ease of setup: Very fast. Template-based deployment means you can launch a pre-configured training environment in minutes. Their pod UI is intuitive.

Network: Secure cloud pods have inter-pod networking. Community cloud pods don’t have reliable inter-pod networking for multi-node jobs — don’t plan multi-node training on community cloud.

Verdict: RunPod is best for cost-sensitive teams running single-node jobs that can tolerate occasional hardware failures. The 4090 instances are an exceptional deal for fine-tuning and small model training — you get consumer-grade hardware at prices that make A100s feel expensive for the task. Use secure cloud if reliability matters; community cloud if you’re doing experiments.

Vast.ai

Vast.ai is similar to RunPod but leans harder into the marketplace model. You’re renting from individual operators in a bidding-style system, with even more price variability.

Available hardware: Similar range to RunPod. Heavy on consumer GPUs (3090, 4090), plus A100s and H100s.

Pricing: Lower ceiling than RunPod on average. RTX 4090 at ~$0.20–$0.35/hr is common. A100s at ~$1.20–$1.60/hr. Prices are set by individual hosts and fluctuate with demand.

Reliability: The weakest among providers reviewed here. Host machines can go offline, have hardware issues, or experience network instability. For any production-adjacent workload, Vast.ai is the wrong choice.

Ease of setup: Similar to RunPod. Docker-based, template-driven, fast to start.

Verdict: Vast.ai is for teams running highly fault-tolerant training jobs — you’re trading reliability for the lowest possible GPU-hour cost. If you have good checkpointing and can afford job restarts, you can get aggressive pricing here. Not appropriate for time-sensitive or production workloads.

Paperspace (now part of DigitalOcean)

Paperspace was acquired by DigitalOcean and has been repositioned as a more developer-friendly GPU cloud. Their Gradient product wraps GPU compute with a managed notebook and experiment tracking experience.

Available hardware:

• H100 (limited availability)
• A100 80GB
• A6000
• RTX 4000 Ada

Pricing:

• H100: ~$5.95/hr
• A100 80GB: ~$3.18/hr
• A6000: ~$1.89/hr

Ease of setup: The best in this list for absolute beginners. Gradient notebooks work immediately. If you’re doing exploratory work or prototyping, Paperspace minimizes friction.

Verdict: Paperspace is for individual contributors doing experimentation, not for teams running production training pipelines. Pricing is not competitive with Lambda or RunPod for equivalent hardware, but the managed experience is genuinely polished.

Together AI

Together AI is not a raw GPU provider — you don’t get SSH access to a GPU. They offer an API for inference on open-source models and dedicated fine-tuning and deployment infrastructure.

What they offer:

• Inference API for open models (Llama, Mistral, Qwen, etc.) at competitive per-token pricing
• Fine-tuning service for popular base models
• Dedicated deployments for teams that need private endpoints

Pricing: Per-token inference at rates that undercut OpenAI for comparable open models. Llama 3.3 70B at ~$1.04/1M tokens and Llama 4 Scout 17B at ~$0.59/1M output tokens are representative data points.

Why it’s in this comparison: If your use case is deploying a fine-tuned open model for inference — and you don’t want to manage GPU infrastructure — Together AI (and alternatives like Fireworks AI, Replicate, Anyscale) is the right abstraction level. The managed inference layer saves significant ops overhead compared to running your own vLLM cluster.

Verdict: Together AI is for teams that need inference at scale on open models and want an API rather than infrastructure. It’s not in competition with the raw GPU providers for training workloads.

Pricing Summary

Hyperscaler pricing includes managed services, IAM, networking infrastructure, and SLAs. The 4–5x delta vs. GPU-native clouds is real, and for long-running training jobs it adds up fast.

Recommendation Matrix

Training large models (30B+ parameters, multi-node)

First choice: CoreWeave — InfiniBand networking, H100 SXM cluster configurations, HPC storage. The cluster interconnect matters when your all-reduce operations are the bottleneck.

Second choice: GCP a3-mega — If you need hyperscaler integration (BigQuery datasets, GCS checkpointing, managed IAM), GCP’s a3-mega tier offers equivalent InfiniBand networking.

Avoid: Vast.ai, RunPod community cloud (no reliable multi-node networking), Paperspace (hardware availability and pricing).

Inference at scale with uptime requirements

First choice: AWS or GCP — Managed instance groups, auto-scaling, SLAs, and existing organizational trust. You’re not optimizing for price, you’re optimizing for “it keeps running.”

Second choice: CoreWeave with a dedicated deployment. They offer Kubernetes-based deployments with better SLAs than you’d get from the self-service tier.

For managed open-model inference without infrastructure: Together AI, Fireworks AI — No infrastructure to manage, competitive per-token pricing.

Budget-conscious teams

Fine-tuning and small model training: RunPod RTX 4090 instances or Lambda A10 instances. You don’t need A100s to fine-tune a 7B or 13B model. A 4090 has 24GB of VRAM and runs QLoRA fine-tuning efficiently.

Single-node A100 training: RunPod secure cloud or Lambda Labs on-demand. RunPod offers A100 80GB at ~$1.64–$1.99/hr; Lambda is ~$2.79/hr but with better reliability and no egress fees.

Fault-tolerant batch jobs: Vast.ai. If you have good checkpointing and the workload can restart, you can get meaningful cost savings vs. any other provider.

Quick experiments and prototyping

RunPod or Paperspace. Both offer instant GPU access via browser, pre-built ML environment templates, and reasonable pricing for short sessions. Paperspace Gradient is more polished; RunPod is cheaper and has more GPU variety.

Lambda Labs is also excellent here if you prefer SSH over Jupyter-style interfaces.

What to Watch For

Network egress costs: Hyperscalers charge for data leaving their network. If you’re pulling large datasets or checkpointing to external storage, egress fees can materially change your cost calculation.

Storage performance: Multi-node training is often I/O bound on dataset reads. NFS-backed storage (common on Lambda, RunPod) is fine for most fine-tuning but struggles with the data throughput required for pretraining at scale. CoreWeave’s Weka filesystem and AWS FSx for Lustre are purpose-built for this.

Spot interruption handling: AWS and GCP spot interruptions come with a 2-minute warning. RunPod and Vast.ai host interruptions are less predictable. Design for interruption with regular checkpointing regardless of provider.

H100 vs. H100: Not all H100 configurations are equal. SXM (high-bandwidth interconnect, shared memory bus) vs. PCIe matters for multi-GPU jobs. The price difference is real; so is the performance difference for communication-heavy workloads.

Reserved vs. on-demand: For any steady-state workload running more than ~40% of the time, reserved capacity (hyperscalers) or committed contracts (CoreWeave, Lambda) will be cheaper than on-demand. Run the math before assuming on-demand is simpler.

The GPU market is moving fast. H100 availability continues to improve with spot rates under $2/hr, H200 instances are now widely available across hyperscalers and GPU-native clouds, and B200/GB200 hardware is deployable at select providers like GMI Cloud and Nebius. Pricing is compressing as supply catches up to demand. The recommendations above reflect mid-2026 market conditions — revisit before signing any long-term contracts.