Parts and reads for a local-AI box
Hardware articles worth your time for 1–4 GPU headless machines, each with a short note from us, and a hand-picked parts catalog traced to community builds and manufacturer specs, with buy links for the US, Germany and the UK.
Latest reads
All reads-
NZXT C1000 Platinum 2026 PSU Review
A 1000W unit with one 12V-2x6 plug and four 8-pin PCIe leads, so a second card can run off the 8-pins. Tight for two full-power 3090-class cards once transients hit: power-limit them, or step up to a 1600W unit.
See PSU, Corsair AX1600i
-
NVIDIA releases Statement on GeForce RTX 5090: GB202 Allocations / 32GB GDDR7 Active Supply
NVIDIA says the RTX 5090 is not discontinued, but AI integrators are buying cards for blower-style servers and partners expect tight stock for about a quarter. That keeps a used RTX 3090 the sane way to buy 24GB today.
-
Introducing the ROCm Handbooks
A new entry point to the ROCm docs. If you are weighing AMD cards for a local AI box, start here before buying.
See GPU
-
AMD EPYC 9006 Results Are Here and AMD Delivers a Knockout Part Deux
Venice moves EPYC to a new SP7 socket with 96 PCIe 6.0 lanes in single-socket form, fewer than SP5's 128. For home builds the takeaway is the older SP3 and SP5 platforms: more lanes per dollar, and more of them reaching the used market.
-
Intel Xeon 600 Workstation Content Creation Review
Intel's new workstation platform has 128 PCIe lanes, enough for several x16 GPUs from one socket. Puget tests creative apps, not AI, but the 24- and 32-core chips are the value picks if you want a new Intel host over used EPYC.
See CPU, Motherboard
-
ROCm InferaSim: Cut LLM Serving Trial Costs and Deploy the Best Configuration
Simulating LLM serving setups before buying or deploying is exactly the question a multi-GPU builder faces: how many cards, which split.
See GPU
-
Supermicro Super AI Station Review: The AI Server That Doubles as a Workstation
A vendor-built answer to the same problem we build by hand: a quiet desk-side AI box. Worth comparing its layout and cooling to a DIY open frame.
See Case/Frame, Motherboard
-
Online Quantization: Smooth Path from One Checkpoint to Any Desired Serving Scenario
Quantizing at load time from one checkpoint means fewer model copies on disk and more freedom to fit the VRAM budget of each box.
-
From Operator to Kernel: Profile DeepSeek-V4-Flash in Primus with madengine
A walk-through of profiling a large model from operator down to kernel on AMD. Deep end of the pool, but good for tracking down slow layers.
See GPU
-
Quark Mixed-Precision Quantization for Wan2.2 Video Generation on AMD Instinct GPUs
Mixed-precision quantization trades a little quality for a lot of memory. Relevant whenever a model is just too big for the VRAM you have.
See GPU
-
Performance Profiling on AMD GPUs - Part 7: AI/PyTorch® Edition
Part 7 of a profiling series, focused on PyTorch. Handy if you run ROCm at home and want to see where a model actually spends its GPU time.
See GPU
-
Fixing the Vertical GPU Problem | HAVN HS 360 Case Review, Thermals, & Benchmarks
Vertical GPU mounts and riser airflow are a real problem once you stack cards. A consumer case, but the thermal lessons carry over to multi-GPU builds.
-
Announcing Humming: Fast, Flexible Kernels for Quantized Inference
A new vLLM kernel library for low-bit weights on NVIDIA cards from Turing onward. Smaller weights let a bigger model fit in the same VRAM; Humming aims to make them fast as well as small. No AMD support yet.
-
NVIDIA Has GPUs If You Have Coin
Context for why used high-VRAM cards stay attractive: new NVIDIA supply goes to whoever pays most. Read it before ruling out a used RTX 3090.
-
Best Professional GPUs For Topaz’s Starlight Models
A video-upscaling GPU test, not an LLM one, but VRAM matters there too. Useful as a cross-check on how pro cards compare per GB of memory.
See GPU
Browse the catalog
- Starter 12 GB build Build tier One used RTX 3060 12 GB added to a desktop you already own, enough for 8B to 14B models at Q4 to Q8 on the full CUDA stack.
- 24 GB build Build tier One used RTX 3090 on the 48 GB build's AM5 platform, for 30B-class models at Q4 today and a second card later.
- 48 GB build Build tier Two used RTX 3090s on an AM5 board at x8/x8, sized to run a 70B model at Q4 entirely in VRAM.
- 96 GB build Build tier Four used RTX 3090s on risers from a single-socket EPYC board, each at x16, fed by two synced PSUs on an open frame.
- GPU 10 parts
- CPU 6 parts
- Motherboard 8 parts
- RAM 5 parts
- Storage 5 parts
- PSU 7 parts
- Case/Frame 6 parts
- Cooler 5 parts
- Risers & Cables 8 parts
- Community Builds Survey Survey What people actually put in multi-GPU local-AI machines, counted from builds their owners documented in public, each linked to its write-up.
- Headless setup for a local-AI box Explainer Running a GPU box without a monitor, from display output and the NVIDIA driver to persistence mode, a power limit at boot, SSH, BIOS settings and serving a model on the LAN.
- Storage for model weights Explainer How much drive space a model library takes, how drive read speed bounds load time, why a PCIe 5 drive buys little, and why endurance is a weak concern for model files.
- PCIe lanes, bifurcation and risers for multi-GPU builds Explainer How many PCIe lanes a multi-GPU inference box needs, how boards wire their slots, and which risers and cable adapters keep a Gen4 link clean.
- Running a local-AI box 24/7, power and noise Explainer How the Build Tiers estimate daily kWh, what idle draw and persistence mode add, why the power limit is the main lever, and what is known about noise.
- PSU sizing and power-limiting for multi-GPU builds Explainer How this site counts the GPUs a PSU supports, why power limits do not stop transient spikes, and what connectors, dual PSUs and the wall circuit add.
- System RAM for offloading models, 128 to 256 GB Explainer When layers or MoE experts spill from VRAM into system RAM, how memory channels and DIMM count set the speed, and how much RAM a model file needs.
- Used RTX 3060 12 GB buying checklist Explainer What to check on a second-hand RTX 3060 12 GB before it goes into a local-AI box, from the listing to the first half hour under load.
- Used RTX 3090 buying checklist Explainer What to check on a second-hand RTX 3090 before it goes into a local-AI box, from the listing to the first half hour under load.
- How much VRAM a local model needs Explainer Sizing VRAM from the model file, the KV cache and runtime overhead, with the arithmetic worked for the 12, 24, 48 and 96 GB builds.