The Starter 12 GB build

Verdict

For someone with a desktop PC already, who wants to try local models without building a new machine. You add one used RTX 3060 12 GB and nothing else. 12 GB holds a 14B model at Q4_K_M with an 8k context, or an 8B model at Q8_0 with 16k. The card is rated at 170 W, and the one Basis build that reports a figure saw its whole machine draw about 280 W under full GPU and CPU load. It is the least expensive way into this site's Tiers, because the only purchase is one used card; the trade-off is speed, since the 3060's memory bandwidth is a fraction of a 3090's.

Model fit

Parts list

Store links may earn us a commission at no cost to you. As an Amazon Associate we earn from qualifying purchases.

Basis

Compatibility warnings

This Tier is one card going into a PC you already own, so every warning here is a check on that host machine.

• A free x16 slot. The card needs a full-length PCIe x16 slot. On a typical desktop that is the top slot; check that nothing else (a Wi-Fi or capture card) already sits in it or right below it. The PCIe lanes and risers guide covers slots and lanes in more depth.
• Length and width. ASUS lists the Dual RTX 3060 V2 OC at 200 mm long and two slots wide. Other partner cards differ, and NVIDIA’s own page gives no length or width for the 3060, so measure for the exact card you buy. Measure height as well: zhonguncle’s Dell T5810 takes long cards, but his 3060 was too tall for the side panel to close, and he ended up mounting it vertically.
• One 8-pin PCIe cable. The ASUS card takes one 8-pin connector. OEM machines can have a single GPU cable, an adapter or a short run: the T5810’s GPU cable is an 8-pin-to-dual-6-pin adapter, and zhonguncle put the card in slot 3 because the cable would not reach further. Check what your PSU has before buying.
• PSU headroom. NVIDIA rates the RTX 3060 at 170 W of graphics card power and asks for a 550 W system power supply. Igor’s Lab measured a partner card peaking at about 175 W, warns that 1 to 10 ms spikes can trip a PSU’s protection even when the average is in spec, and suggests budgeting at least 230 to 250 W for the card’s share. The two Basis machines bracket what works: the Precision 3630 runs the card on its stock 460 W 80+ Gold Dell PSU, which sahibgasimov reports “handles the 3060 fine” at about 280 W total under full GPU and CPU load. zhonguncle’s T5810 runs an 825 W unit, after the seller first shipped a 425 W one. If your PSU is below NVIDIA’s 550 W, check its 12 V rating against the card plus your CPU before you buy. The PSU sizing guide shows how to redo this site’s PSU arithmetic for a 170 W card.
• System RAM. Both Basis machines have at least 32 GB (32 GB in the T5810, 48 GB in the 3630). Aim for about that: the model file has to load through system memory, and anything that doesn’t fit on the card spills into it. The system RAM guide explains how offloading to RAM works and how memory bandwidth sets its speed.
• Display output. Each GB the desktop uses on the 3060 is a GB the model can’t. If your CPU has integrated graphics and the board has a video output, plug the monitor in there and keep the 3060 for models; the Model fit margins assume that. If it has none, the 3060 (one HDMI and three DisplayPort outputs per NVIDIA) also drives the screen, so leave more margin than the Model fit rows do. The headless setup guide covers running the box without a monitor once it is set up.
• Household circuit. About 280 W for the whole machine is a small load: a US 15 A circuit at 120 V carries 1800 W, and a continuous load should stay under 80 % of that, 1440 W. On 230 V supplies (a UK plug’s 13 A fuse, about 3.0 kW, or a 16 A Schuko socket, about 3.7 kW) the load is smaller still against the rating. This Tier needs no electrical planning.
• Buying used. Read the used RTX 3060 checklist first. It covers the 8 GB card sold under the same name, which is not this Tier.

Power & noise

Estimated 24/7 energy use, with every assumption stated:

• Load: about 280 W for the whole machine, the figure sahibgasimov reports for his Precision 3630 with the 3060 and the CPU both at maximum load. That is an upper bound: the same page reports the card at 30 to 80 W in normal use with a 9B model and 170 W at its peak, so typical daily use is likely lower. His page does not say whether it was measured at the wall; this estimate treats it as a wall figure. A faster host CPU would draw more.
• Idle: Igor’s Lab measured about 12 W for a partner RTX 3060 at idle (and blames part of that on its RGB lighting). The rest of the host at idle is an assumption, not a measurement: 50 W for board, CPU, RAM, storage and fans on an older desktop. That gives about 62 W delivered by the PSU.
• At the wall (idle): the Basis 3630’s PSU is 80+ Gold, which requires 87 % efficiency at 20 % load on 115 V. This estimate assumes 87 %, so 62 W ÷ 0.87 is about 71 W. At this load the PSU runs near or below 20 %, so the real figure may be somewhat higher.
• Duty cycle: an assumption for a personal inference box: 4 hours of generation and 20 hours idle each day.

Arithmetic: 4 h × 0.280 kW ≈ 1.1 kWh, plus 20 h × 0.071 kW ≈ 1.4 kWh, gives about 2.5 kWh a day at the wall. That is about 77 kWh a month (× 30.4 days) and about 930 kWh a year (× 365). Most of it is the host idling, not the card working, so the cheapest saving is to sleep or shut the machine down when you’re not using it. Multiply by your own tariff. The power and noise guide explains the method and how to redo it for your own hours.

Noise: we have no noise measurement for either Basis build. sahibgasimov reports the card at 50 to 60 °C and 30 to 80 W under normal load with a 9B model, peaking around 75 °C and 170 W under sustained inference. The ASUS card is a two-fan open-air design, so its heat stays in the case; check that the host has at least one working intake and one exhaust fan. zhonguncle returned an RTX 2080 Ti with a blower cooler for coil whine before settling on the 3060, so listen to a used card under load while you can still return it.

FAQ

Has anyone built exactly this? The exact card and host pairing, no. Both Basis builds are upgraded OEM machines with a used 12 GB RTX 3060 added, which is this Tier’s whole idea: zhonguncle’s Dell T5810 (Xeon E5-2680 v4, 32 GB, 825 W) and sahibgasimov’s Dell Precision 3630 (Core i7-9700, 48 GB, stock 460 W). Neither uses the ASUS Dual V2 OC this site lists (sahibgasimov’s is an EVGA XC), and neither reports a tokens-per-second figure, so every Model fit row is our arithmetic.

How was the fit worked out? The VRAM sizing guide explains the method. The card holds 12,288 MiB, about 12.88 GB. Qwen2.5 14B has 48 layers, 8 key/value heads and a head size of 128 (5,120 ÷ 40 heads). A 16-bit KV cache, llama.cpp’s default, therefore takes 48 × 8 × 128 × 2 (keys and values) × 2 bytes = 196,608 bytes per token. At 8,192 tokens that is about 1.61 GB; with the 8.99 GB Q4_K_M file, about 10.6 GB, leaving roughly 2.3 GB for the CUDA context and compute buffers. At 16k the cache is 3.22 GB and the margin falls to about 0.7 GB. Llama 3.1 8B (32 layers, 8 key/value heads, head size 128) takes 131,072 bytes per token, 2.15 GB at 16k and 4.29 GB at 32k. Phi-4 (40 layers, 10 key/value heads, head size 128) takes 204,800 bytes per token, 1.68 GB at 8k and 3.36 GB at 16k.

Why a 3060 and not a newer 8 GB card? 12 GB is the point where a 14B model at Q4 fits with context to spare, and bought used, the 3060 is the least expensive card that gets there on CUDA. Its weak point is memory bandwidth: 360 GB/s against the RTX 3090’s 936 GB/s, so expect it to be slower than the bigger cards on the same model.

What’s the upgrade path? The 24 GB build. It is a new machine around one used RTX 3090, sized from the start to take a second card. 24 GB moves you from 14B models to 30B-class models at Q4. The 3060 is not on that Tier’s parts list; keep it in the old PC or sell it.

Can I put two 3060s in my PC instead? Possibly, for 24 GB, but most OEM hosts are the limit. You need a second x16-length slot with room for a two-slot card, a second 8-pin cable and more PSU headroom. zhonguncle calls dual GPUs in his T5810 “risky” for exactly those reasons. If your host can’t take it, the 24 GB build is the cleaner step.