NVIDIA's H100 product table distinguishes H100 SXM from H100 NVL. They have different memory and power specifications. This guide compares the 80 GB SXM configuration with H200 SXM, not the 94 GB H100 NVL product. A provider listing that only says H100 does not establish which row of that table applies. [1]

We recommend preserving the original provider wording until the variant is documented. A broad H100 family listing can still be useful for sourcing, but it should not silently inherit the most favorable specification from another variant. The same rule applies to host networking and GPU quantity. This article is an architectural comparison, not a claim that any particular provider currently has either configuration available.

The manufacturer lists H100 SXM with 80 GB of GPU memory and 3.35 TB/s memory bandwidth. Its H200 table lists 141 GB HBM3e and 4.8 TB/s for H200 SXM. NVIDIA qualifies the H200 specification table as preliminary and subject to change. These are published device specifications, not independently measured sustained rates for a rented server. [1][2]

The manufacturer's SXM tables list configurable power envelopes up to 700 W for these particular configurations. Equal maximum device power does not imply equal server power, equal sustained operating draw or equal energy per completed job. Those depend on the allocation, operating settings, runtime and supporting hardware. [1][2]

The smaller-memory option might become viable through quantization, lower concurrency, offloading or model distribution. But that would be a changed implementation. Our recommendation is to compare both the unchanged workload and the feasible alternative designs. Otherwise a benchmark can incorrectly attribute a large gain to raw processing speed when the real benefit is avoiding offloading or eliminating a cross-device boundary.

NVIDIA's performance documentation distinguishes arithmetic-limited operations from operations limited by data movement. Arithmetic intensity relates useful operations to bytes transferred through a specified memory boundary. A workload with insufficient reuse can wait for memory even when arithmetic resources are available. A specification comparison should therefore identify which boundary limits the workload before assigning value to a higher peak rate. [3]

Autoregressive serving can maintain a KV cache whose memory demand changes with architecture, context and active requests. Cache strategies can trade memory against runtime behavior. Successfully loading a model is consequently not evidence that it can serve the desired number of long-context requests. The workload description should specify prompt and output distributions as well as model identity. [4]

Report time to first token, output-token pacing and aggregate throughput separately. NVIDIA's GenAI-Perf documentation defines metrics for these different boundaries. A higher aggregate token rate does not establish a faster response for every user. The acceptance target should state which latency percentiles and output-quality criteria the service must satisfy. [5]

Moving from one GPU to multiple GPUs adds placement and communication decisions. A framework can replicate or partition model state, and communication libraries perform collective operations over the available paths. The performance of those operations depends on the supplied topology. An eight-GPU rental cannot be characterized only by multiplying a single-device memory figure by eight. [6]

Now suppose the second allocation instead takes 3.5 hours. Its cost becomes $12.25, reversing the ranking. The break-even runtime ratio follows from the price ratio when other charges are equal. Real contracts can add minimum allocations, storage, transfers and commitments. We recommend applying actual quoted terms to measured workload durations, rather than treating a catalogue price leader as a universal economic winner.

Start with exact variant and software compatibility. Estimate and then measure memory at intended concurrency. Test the same accepted workload, record the environment and report latency as well as throughput. Include required GPU count and communication topology. Finally, compare full commercial terms and the time at which capacity can actually be supplied. This sequence makes a recommendation conditional and reproducible.

H200's published memory specifications can be especially relevant when H100's capacity or memory path is the binding constraint. That is a mechanism to investigate, not a blanket claim that H200 wins every task. When an existing H100 configuration already meets the objective at lower total cost, the larger specification may not justify a switch. The right answer belongs to the workload and allocation, not the model name alone.