The 48 GB build

Verdict

For a builder who wants a 70B model at Q4 fully in VRAM on one desktop box. Two used RTX 3090s give 48 GB, enough for Llama 3.3 70B at Q4_K_M with an 8k context, and it is the cheapest path to 48 GB because both cards are bought used. The cost is power: each card is rated at 350 W, and the one Basis build with a measured wall draw peaked at 1.04 kW, so plan on power-limiting both cards and on a circuit with nothing else heavy on it.

Model fit

Parts list

Store links may earn us a commission at no cost to you. As an Amazon Associate we earn from qualifying purchases.

Basis

Compatibility warnings

• Slot spacing. The Founders Edition RTX 3090 is three slots wide and 313 mm long. Measured to scale from the layout drawing in ASRock’s manual (PDF page 11), the Taichi’s PCIE1 and PCIE2 sit about 80 mm apart, four slot positions. Each card fills three of those, so two Founders Editions seat directly with one empty slot position, roughly 20 mm, of air between them. That gap is why this Tier uses the Taichi and not a board with three-slot spacing, where the cards touch. den.dev hit exactly that on an X570 board: “if you have two 3-slot cards, they will literally be touching each other, with no gap for air”, and moved one card to a vertical mount on a 45 cm riser. hav4ik went to an open-air frame for the same reason. A partner card thicker than three slots closes the gap, so check its width. The PCIe lanes and risers guide covers slot spacing and lanes across boards.
• No risers. Both cards go straight into the board. That removes the riser as a failure point and a cost line. You need risers only if you move to a frame or layout that doesn’t line the cards up with the board’s slots.
• The second card overhangs the board. PCIE2 is the Taichi’s last slot, so the lower card hangs two slot positions past the board’s bottom edge. The Define 7 XL has nine expansion slots and takes E-ATX boards, so there are openings below the board for it. Check any other case for both.
• Phanteks alternate. The Enthoo Pro 2 Server Edition has 11 expansion slots, takes E-ATX boards, and its 503 mm of GPU clearance clears the 313 mm Founders Edition with room to spare. Its included support bracket holds full-length cards, which helps with two 3090s hanging off the board.
• Lanes. With both slots filled, ASRock’s manual runs them at x8/x8 from the CPU. The RTX 3090 is a PCIe 4.0 card, so each card gets 8 lanes of PCIe 4.0. The Ryzen 9 9950X’s 24 usable lanes cover the two GPUs plus the CPU-attached M2_1 slot. The manual’s block diagram shows no M.2 slot sharing lanes with the GPU slots on this board: M2_1 hangs off the CPU and M2_2 to M2_4 off the chipset. That differs from the ASUS ProArt X870E-Creator, where filling its second M.2 slot drops the second GPU slot to x4.
• Power connectors. NVIDIA’s Founders Edition takes two 8-pin PCIe cables through the 12-pin adapter in the box, so two cards need four 8-pin cables. Run one dedicated cable from the PSU to each 8-pin input; don’t feed two inputs from one cable’s daisy-chained pigtail. The HX1200i and HX1500i each list five 8-pin PCIe connectors, and their 12V-2x6 cables share the same sockets, so you can’t use all of both counts. A partner card with three 8-pin inputs would need six for the pair, one more than either PSU lists. Count connectors on the exact cards you buy.
• Transient spikes and power-limiting. NVIDIA rates the RTX 3090 at 350 W and asks for a 750 W PSU for a single-card system. Igor’s Lab warns that the card’s short load peaks, at intervals of 1 to 10 ms, can trip a PSU’s protection and shut the system down. hav4ik reports that the cards’ “power draw sometimes spikes up to 600W for brief periods” and bought a 1600 W PSU for that reason. This site’s PSU estimate gives the HX1200i two GPUs: 1200 W × 0.8, minus 250 W for the rest of the system, divided by 300 W per GPU, rounded down. That assumes power-limited cards, so set a limit with nvidia-smi -pl 300 on each card (it needs root). The limit does not survive a reboot or a driver reload, so reapply it at boot: Puget Systems does it with a systemd service that turns on persistence mode (nvidia-smi -pm 1) and then sets the limit. A power limit caps average draw; it does not cap the millisecond transients above. Unlimited, the same arithmetic gives 2 × 350 W + 250 W = 950 W against the HX1200i’s 960 W, with almost nothing left for spikes. If you won’t power-limit, or want headroom for the transients anyway, take the HX1500i, which the same estimate gives three GPUs. The PSU sizing guide explains the estimate and its limits.
• The HX1500i on low-voltage mains. Corsair rates the HX1500i at 1500 W from 115 V up but 1200 W on 100 to 115 V input. On that input it has no more headroom than the HX1200i.
• Household circuit. A US 15 A circuit at 120 V carries 1800 W, and a continuous load (one running more than three hours) should stay under 80 % of that, which is 1440 W. jonready measured his two-3090 machine peaking at 1.04 kW at the wall under a parallel benchmark. That fits, but it leaves about 400 W for everything else on the same circuit. Don’t share it with a space heater or a second machine. On 230 V mains there is more room: a UK plug takes at most a 13 A fuse, about 3.0 kW (check the fuse actually fitted in your PC’s plug), and a German Schuko socket is rated 16 A, about 3.7 kW. Keeping the same 80 % margin still leaves well over 1 kW spare, but the circuit behind the socket is shared with other sockets, so the same advice about heaters applies.
• Keep both 3090s headless. The VRAM margins in the FAQ assume no desktop is running on either card. The Ryzen 9 9950X has integrated AMD Radeon graphics, so plug the monitor into the Taichi’s own HDMI or USB4 outputs and leave both 3090s free for models. The headless setup guide covers the rest of running it without a monitor.
• Upper card’s memory. Half of each 3090’s GDDR6X sits on the back of the board, cooled only by the backplate. In this layout the lower card’s heat rises into the upper card’s backplate, so watch its memory junction temperature under sustained load as the used RTX 3090 checklist describes.
• CPU cooler. The NH-D15 G2 is 168 mm tall; Fractal’s product sheet gives the Define 7 XL 185 mm of cooler height, and Phanteks lists 195 mm for the Enthoo Pro 2 Server Edition. On the Taichi the M2_1 slot sits between the socket and PCIE1 (manual layout, PDF page 11), so the top card’s slot is not directly under the cooler. Noctua’s compatibility list for the X870E Taichi includes the NH-D15 G2 with no extra mounting kit and no top-PCIe-slot restriction noted. What it doesn’t settle is RAM height under the front fan, so check the RAM kit’s height against Noctua’s NH-D15 G2 specification before ordering.

Power & noise

Estimated 24/7 energy use, with every assumption stated:

• Load: both cards power-limited to 300 W (the per-GPU figure in this site’s PSU estimate), plus 250 W for the rest of the system (the same estimate’s system reserve). That gives 2 × 300 W + 250 W = 850 W delivered by the PSU while generating.
• Idle: Igor’s Lab measured about 16 W for one RTX 3090 at idle, so 32 W for two. The rest of the system at idle is an assumption, not a measurement: 70 W for the board, CPU, RAM, SSD and fans. That gives about 100 W delivered by the PSU at idle.
• At the wall: the HX1200i is 80+ Platinum, which requires 89 to 92 % efficiency between 20 % and 100 % load on 115 V. This estimate assumes 90 %, so divide by 0.9: about 944 W at load and 111 W at idle. At idle the PSU runs below 10 % load, where 80 Plus sets no Platinum figure, so the real idle draw is probably somewhat higher.
• Duty cycle: an assumption for a personal inference box: 4 hours of generation and 20 hours idle each day.

Arithmetic: 4 h × 0.944 kW ≈ 3.8 kWh, plus 20 h × 0.111 kW ≈ 2.2 kWh, gives about 6.0 kWh a day at the wall. That is about 182 kWh a month (× 30.4 days) and about 2,190 kWh a year (× 365). For comparison, jonready’s measured 1.04 kW peak at the wall, held around the clock, would be 24 h × 1.04 kW ≈ 25 kWh a day, about four times as much. Multiply by your own tariff; the duty cycle moves the result far more than any part choice here. The power and noise guide explains the method and how to redo it for your own hours.

Noise: the two 3090s are the loud part. Each Founders Edition cooler moves air through the card, and with only about one slot position of air between them, the lower card blows hot air into the upper one. hav4ik names that as the reason closely stacked 30-series cards overheat and throttle. Power-limiting to 300 W lowers both heat and fan speed. The Define 7 XL is sound-damped, but the damping also holds heat, so fill the front and top fan mounts. The Phanteks alternate has 15 fan positions, so it can move more air through the two cards; we have no noise measurement for either case.

FAQ

Has anyone built exactly this list? Not that we know of. The three Basis builds share its core: two RTX 3090s in one desktop, run for local models. Their boards differ (an X570 pair and a TRX40 Threadripper), and jonready’s cards are NVLink-bridged. Treat the measured Model fit rows as evidence for the 2× 3090 configuration, not for these exact parts.

Why two used 3090s and not one newer card? The RTX 3090 is the cheapest way to 24 GB per card when bought used, and two of them reach 48 GB. Read the used RTX 3090 checklist before buying, and test each card under load while you can still return it.

How was the 70B fit worked out? The VRAM sizing guide explains the method. Llama 3.3 70B has 80 layers, 8 key/value heads and a head size of 128. A 16-bit KV cache therefore takes 80 × 8 × 128 × 2 (keys and values) × 2 bytes = 327,680 bytes per token. At 8,192 tokens that is about 2.68 GB. The Q4_K_M file is 42.52 GB, so weights plus cache come to about 45.2 GB. Two cards hold 2 × 24,576 MiB, about 51.5 GB, which leaves roughly 6 GB for the CUDA context, compute buffers and uneven splitting across the two cards. At 16k the cache doubles to about 5.37 GB and the margin drops to about 3.6 GB. That may work, but it is borderline and untested here. The same arithmetic for Qwen2.5 32B (64 layers, 8 key/value heads, head size 128) gives 262,144 bytes per token, about 8.6 GB at 32k.

Do I need NVLink? Not for llama.cpp. Its default split mode, layer, splits layers and KV cache across the GPUs, so each card holds whole layers and only the hidden state crosses between cards: for Llama 3.3 70B that is 8,192 values, about 16 KB per token at 16-bit, which x8 PCIe 4.0 moves without strain. A bridge matters more for vLLM tensor parallelism, where the cards exchange data inside every layer. jonready’s vLLM numbers were measured with the cards NVLink-bridged on a Threadripper board wired x16/x8, so expect this x8/x8 build without a bridge to land below his multi-request figures; the single-stream figures are the safer guide. If you do add one: NVIDIA sold the GeForce RTX 3090 bridge only in 4-slot spacing, which matches the Taichi’s four-position slot pitch. The 3-slot RTX A6000 bridge, which Puget Systems found works on 3090s, is too short for this board.

Why the Taichi and not a Threadripper board? Two GPUs is all AM5 does, and the Taichi does it at x8/x8 with the cards seated directly. More GPUs or more than 256 GB of RAM means Threadripper or EPYC, which is the 96 GB Tier’s ground.

Why 128 GB of system RAM? It leaves room to load models from disk and to offload layers that don’t fit in VRAM. The kit fills both channels with two DIMMs and leaves two slots for a second kit up to the board’s 256 GB limit, though four DIMMs on AM5 run much slower. The system RAM guide covers offloading and DIMM count, and the model storage guide covers loading from disk.