Tools · AI infrastructureTools · Infrastruktur AI
AI Hardware Sizer
“We want AI in our company. Which GPU and server do we need?” Answer a few questions and get a starting point for GPU, server, CPU, memory and storage. Unfamiliar terms are explained in the glossary.“Kami mau pakai AI di perusahaan, butuh GPU dan server seperti apa?” Jawab beberapa pertanyaan dan dapatkan titik awal untuk GPU, server, CPU, memori, dan storage. Istilah yang asing dijelaskan di glosarium.
Everything runs in your browser; nothing you enter is sent anywhere. GPU data from HPE QuickSpecs (NVIDIA v85, Nov 2026; AMD v20, Aug 2026; HPE Private Cloud AI v12, Nov 2026; DL384 Gen12 v18, Oct 2026; GB200 NVL4, GB200 NVL72 and GB300 NVL72, Sep 2026). Results are estimates, not a quotation.Semua dihitung di browser Anda; tidak ada data yang dikirim ke mana pun. Data GPU dari HPE QuickSpecs (NVIDIA v85, Nov 2026; AMD v20, Agu 2026; HPE Private Cloud AI v12, Nov 2026; DL384 Gen12 v18, Okt 2026; GB200 NVL4, GB200 NVL72, dan GB300 NVL72, Sep 2026). Hasilnya estimasi, bukan penawaran harga.
NotesCatatan
GPU memory neededKebutuhan memori GPU
Compare all GPUsBandingkan semua GPU
StorageStorage
GPUs in this sizer (from the HPE QuickSpecs)GPU di sizer ini (dari HPE QuickSpecs)
From the HPE QuickSpecs. Performance figures are as listed by HPE; some are quoted with sparsity and in different precisions, so do not compare them one to one. The sizer ranks GPUs by memory and memory bandwidth instead.Dari HPE QuickSpecs. Angka performa sesuai yang dicantumkan HPE; sebagian dikutip dengan sparsity dan presisi berbeda, jadi jangan dibandingkan satu lawan satu. Sizer memeringkat GPU berdasarkan memori dan bandwidth memori.
| GPU | MemoryMemori | Bandwidth | PowerDaya | Performance (datasheet)Performa (datasheet) | Interface |
|---|
Not used for sizing: NVIDIA A16 (virtual desktops), RTX A400 and RTX A1000 (display and light graphics), and L20 (the datasheet limits its sale to China, Hong Kong and Macau).Tidak dipakai untuk sizing: NVIDIA A16 (virtual desktop), RTX A400 dan RTX A1000 (display dan grafis ringan), serta L20 (datasheet membatasi penjualannya ke China, Hong Kong, dan Makau).
Rack-scale systems (reference)Sistem skala rack (referensi)
Liquid-cooled systems for very large models. The sizer calculates them separately and shows them only when a workload is very large; they are never the starting recommendation. GPU memory and bandwidth come from NVIDIA's specification pages, because the HPE QuickSpecs do not list them.Sistem berpendingin cairan untuk model sangat besar. Sizer menghitungnya terpisah dan hanya menampilkannya saat beban kerja sangat besar; sistem ini tidak pernah menjadi rekomendasi awal. Memori dan bandwidth GPU diambil dari halaman spesifikasi NVIDIA, karena QuickSpecs HPE tidak memuatnya.
| SystemSistem | Per rackPer rack | GPU memory (NVIDIA)Memori GPU (NVIDIA) | NVLink domainDomain NVLink | PowerDaya | CoolingPendinginan | OS | Built for (QuickSpecs)Dirancang untuk (QuickSpecs) |
|---|---|---|---|---|---|---|---|
| NVIDIA GB300 NVL72 by HPE v3 · 8 Sep 2026 | 18 compute trays: 36 Grace CPUs, 72 Blackwell Ultra GPUs18 compute tray: 36 CPU Grace, 72 GPU Blackwell Ultra | 20 TB total, ± 278 GB per GPU; up to 576 TB/s (NVIDIA)Total 20 TB, ± 278 GB per GPU; hingga 576 TB/s (NVIDIA) | 72 GPUs (9 NVLink switch trays)72 GPU (9 NVLink switch tray) | TDP 132 kW; peak ≈ 155 kW; provision 192 kWTDP 132 kW; puncak ≈ 155 kW; siapkan 192 kW | Direct liquid; 1 CDU (1.3 MW) per up to 8 racksDirect liquid; 1 CDU (1,3 MW) untuk hingga 8 rack | Ubuntu 24.04 | Training, fine-tuning and real-time inference of models above 1 trillion parametersTraining, fine-tuning, dan inference real-time model di atas 1 triliun parameter |
| NVIDIA GB200 NVL72 by HPE v6 · 8 Sep 2026 | 18 compute trays: 36 Grace CPUs, 72 Blackwell GPUs18 compute tray: 36 CPU Grace, 72 GPU Blackwell | 13.4 TB total, 186 GB per GPU; 576 TB/s (NVIDIA)Total 13,4 TB, 186 GB per GPU; 576 TB/s (NVIDIA) | 72 GPUs (9 NVLink switch trays)72 GPU (9 NVLink switch tray) | TDP 132 kW; peak ≈ 192 kWTDP 132 kW; puncak ≈ 192 kW | Direct liquid; 1 CDU (1.3 MW) per up to 8 racksDirect liquid; 1 CDU (1,3 MW) untuk hingga 8 rack | Ubuntu 24.04 | Training, fine-tuning and real-time inference of models above 1 trillion parametersTraining, fine-tuning, dan inference real-time model di atas 1 triliun parameter |
| NVIDIA GB200 NVL4 by HPE v5 · 8 Sep 2026 | Up to 34 trays: 68 Grace CPUs, 136 Blackwell GPUsHingga 34 tray: 68 CPU Grace, 136 GPU Blackwell | Assumed 186 GB per GPU, as in the GB200 superchip (no NVIDIA page for NVL4)Diasumsikan 186 GB per GPU, seperti superchip GB200 (tidak ada halaman NVIDIA untuk NVL4) | 4 GPUs per board pair (PCB bridge)4 GPU per pasangan board (PCB bridge) | Up to 8 power shelves of 33 kWHingga 8 power shelf 33 kW | Direct liquid; 1 CDU (1.3 MW) per up to 4 racksDirect liquid; 1 CDU (1,3 MW) untuk hingga 4 rack | Ubuntu 24.04, RHEL 9.6 | Converged HPC and AI, scientific simulation, model trainingHPC dan AI terpadu, simulasi ilmiah, training model |
How the sizer calculatesCara sizer menghitung
- Model weights = parameters × bytes per precision (FP16 = 2, FP8 = 1, FP4 = 0.5). A 70B model in FP8 needs about 70 GB.
- KV cache = concurrent sessions × half the context × KV size per token for that model class (128 KB for 8B up to 320 KB for 70B, at FP16; halved at FP8/FP4).
- GPU memory needed = (weights + KV cache) × 1.2 for runtime overhead and headroom.
- Recommendation: fewest servers first, then the least total GPU memory (a rough stand-in for cost, since the datasheets carry no prices), then the lower GPU class.
- Speed per user ≈ 60% × total memory bandwidth ÷ (weights + KV cache). Target: 15 tokens/s, faster than people read. If not met, the sizer adds GPUs or model copies.
- Model copies use 1, 2, 4 or 8 GPUs inside one server. HA "keep running": every model runs at least two copies on at least two servers, each sized for its share of peak load, so capacity drops when a server fails. HA "N+1": one extra server of spare capacity, so capacity stays at 100% when a server fails. Video analytics follows the same HA choice. How HA works.
- Other workloads (assumptions): OCR 20 GB per worker per 10,000 pages/day; transcription 10 GB per worker per 200 audio hours/day; image generation 20 GB per concurrent image; video analytics 32 cameras per entry GPU.
- Fine-tuning on dedicated GPUs: QLoRA ≈ parameters × 0.75 + 8 GB; LoRA ≈ parameters × 2.6 + 8 GB; full ≈ parameters × 17.6 GB.
- GH200 (DL384 Gen12): CPU cores and LPDDR5X memory are fixed per superchip (72 Arm cores, 480 GB). Power per superchip is estimated from the QuickSpecs maximum heat dissipation of 3,048 W for a dual-superchip server.
- Server (assumption): up to 2 double-wide or 4 single-slot GPUs in a 2U general server, up to 8 double-wide in a GPU-optimized server (4 for L40S, 4 for MI350P). CPU ≈ 16 + 12 cores per GPU; RAM ≈ 2× GPU memory per server, at least 128 GB. Power ≈ GPU watts + 800 W per server.
- Storage (assumptions): model repository 3× weights; RAG 3× document size; CCTV 2 Mbps per camera (about 21.6 GB per camera per day); scanned pages 0.5 MB; audio 60 MB per hour; archives kept 3 years, 250 working days a year.
- Confirm GPU counts per server and every final configuration against the server QuickSpecs before quoting.
- Bobot model = jumlah parameter × byte per presisi (FP16 = 2, FP8 = 1, FP4 = 0,5). Model 70B di FP8 butuh sekitar 70 GB.
- KV cache = sesi bersamaan × setengah konteks × ukuran KV per token untuk kelas model tersebut (128 KB untuk 8B hingga 320 KB untuk 70B, di FP16; setengahnya di FP8/FP4).
- Memori GPU dibutuhkan = (bobot + KV cache) × 1,2 untuk overhead runtime dan cadangan.
- Rekomendasi: server paling sedikit dulu, lalu total memori GPU terkecil (pendekatan kasar untuk biaya, karena datasheet tidak memuat harga), lalu kelas GPU yang lebih rendah.
- Kecepatan per user ≈ 60% × total bandwidth memori ÷ (bobot + KV cache). Target: 15 token/detik, lebih cepat dari kecepatan orang membaca. Jika belum tercapai, sizer menambah GPU atau salinan model.
- Salinan model memakai 1, 2, 4, atau 8 GPU dalam satu server. HA "tetap jalan": setiap model berjalan minimal dua salinan di minimal dua server, masing-masing dihitung untuk porsi beban jam sibuknya, sehingga kapasitas turun saat satu server mati. HA "N+1": satu server tambahan sebagai kapasitas cadangan, sehingga kapasitas tetap 100% saat satu server mati. Analitik video mengikuti pilihan HA yang sama. Cara kerja HA.
- Beban kerja lain (asumsi): OCR 20 GB per worker per 10.000 halaman/hari; transkripsi 10 GB per worker per 200 jam audio/hari; pembuatan gambar 20 GB per gambar bersamaan; analitik video 32 kamera per GPU entry.
- Fine-tuning di GPU khusus: QLoRA ≈ parameter × 0,75 + 8 GB; LoRA ≈ parameter × 2,6 + 8 GB; full ≈ parameter × 17,6 GB.
- GH200 (DL384 Gen12): core CPU dan memori LPDDR5X tetap per superchip (72 core Arm, 480 GB). Daya per superchip diestimasi dari disipasi panas maksimum 3.048 W di QuickSpecs untuk server dual-superchip.
- Server (asumsi): hingga 2 GPU double-wide atau 4 GPU single-slot di server umum 2U, hingga 8 GPU double-wide di server GPU-optimized (4 untuk L40S, 4 untuk MI350P). CPU ≈ 16 + 12 core per GPU; RAM ≈ 2× memori GPU per server, minimal 128 GB. Daya ≈ watt GPU + 800 W per server.
- Storage (asumsi): repositori model 3× bobot; RAG 3× ukuran dokumen; CCTV 2 Mbps per kamera (sekitar 21,6 GB per kamera per hari); halaman scan 0,5 MB; audio 60 MB per jam; arsip disimpan 3 tahun, 250 hari kerja per tahun.
- Konfirmasi jumlah GPU per server dan setiap konfigurasi akhir ke QuickSpecs server sebelum membuat penawaran.
Want a second opinion on your AI plan?Butuh pendapat kedua untuk rencana AI Anda?
I am open for consultation: ping me with your result and we will turn it into a configuration that fits your data center.Saya terbuka untuk konsultasi: kirimkan hasil sizer Anda, lalu kita jadikan konfigurasi yang pas untuk data center Anda.