1. Product Overview
The NVIDIA H200 (codename: GH200) is the final flagship GPU based on the Hopper architecture, unveiled by NVIDIA at SC23 in 2023 and launched for mass production in Q2 2024. It is the world’s first GPU equipped with HBM3e memory.
Positioning: Primary accelerator for 7B–70B LLM training, high-concurrency inference for 13B–70B large language models, the preferred upgrade solution for H100 clusters, and the most cost-effective mainstream flagship GPU in the current market.
Architecture: Hopper (enhanced version of GH100, single-die design)
Process Technology: TSMC 4nm process
Memory: 141GB HBM3e (double the memory capacity of H100)
Form Factor: Dual versions including SXM5 liquid-cooled model and PCIe air-cooled model
Product Position: Predecessor of B200/B300 series, and a direct upgrade and replacement for the H100 GPU
2. Detailed Specifications of Single H200 GPU
2.1 Architecture & Process Technology
Architecture: Hopper (GH200, single-die)
Process Technology: TSMC 4nm
Transistor Count: Approximately 80 billion
2.2 Computing Cores
Streaming Multiprocessors (SMs): 132
CUDA Cores: 16,896
4th-Generation Tensor Cores: 528
Supported Precision Formats: FP8, FP16, BF16, TF32 (FP4 is not supported)
2.3 Memory System
Memory Capacity: 141GB HBM3e
Memory Bus Width: 8192-bit
Memory Bandwidth: 4.8 TB/s
ECC Support: Full-capacity ECC enabled
MIG Partitioning: Maximum 7 independent instances (18GB memory per instance)
2.4 Single-Card Dense Computing Performance
FP8 (Training): 3.94 PFLOPS
FP16/BF16: 1.97 PFLOPS
INT8: 3.94 TOPS
2.5 Interconnection & Interfaces
NVLink 4.0: 450GB/s unidirectional bandwidth, 900GB/s bidirectional bandwidth
PCIe Gen5: 128GB/s bandwidth
Interface Types: SXM5 (liquid-cooled) / PCIe (air-cooled)
2.6 Power Consumption & Thermal Dissipation
Total Graphics Power (TGP): 700W (SXM version) / 600W (PCIe version)
Cooling Solution: Liquid cooling for SXM model; air cooling / cold plate cooling for PCIe model
3. DGX H200 (8-GPU Server) Full Specifications
3.1 Hardware Configuration
GPU Configuration: 8 × H200 SXM5 GPUs, with a total HBM3e memory of 1,128GB
CPU Configuration: Dual Intel Xeon Platinum 8480C processors (56 cores per CPU)
System Memory: 2TB DDR5 (expandable up to 4TB)
NVSwitch: 4 × 4th-generation NVSwitch, with aggregated bandwidth of 7.2TB/s
Network Configuration: 10 × ConnectX-7 400G InfiniBand / Ethernet ports
Storage Configuration: 8 × E1.S 3.84TB NVMe SSDs (total storage of 30.72TB)
Total System Power Consumption: Approximately 8kW
Chassis Specification: 10U rack-mounted, liquid-cooled chassis
3.2 Overall System Computing Performance
FP8 Training Performance: 31.5 PFLOPS
FP16 Computing Performance: 15.7 PFLOPS
INT8 Computing Performance: 31.5 TOPS
____________________________________________________________
NVIDIA B300
NVIDIA B200
NVIDIA H200
NVIDIA H100