Running a local-AI box 24/7, power and noise

A box that runs around the clock spends most of its day doing nothing, and that is where much of its energy goes. Every Build Tier on this site states its daily kWh with the assumptions spelled out, so you can redo the sum for your own hours and your own tariff. This guide explains the method, shows where it is weakest, and covers the two things you can change: the idle draw and the power limit. It ends with what is known about noise, which is not much.

How the Tiers estimate kWh

Each Tier’s Power & noise section runs the same five steps:

1. Load, delivered by the PSU. For the 3090 Tiers: each card at its 300 W power limit, plus the 250 W system reserve from this site’s PSU estimate. The Starter build uses the whole-machine figure its Basis builder reports instead.
2. Idle, delivered by the PSU. Igor’s Lab’s measured idle for the card (about 16 W for an RTX 3090, about 12 W for a partner RTX 3060), plus an assumed figure for the rest of the machine: 50 W for the Starter host, 70 W for the AM5 platform, 100 W for the EPYC platform. Those platform figures are assumptions, not measurements.
3. Divide by PSU efficiency to get the draw at the wall. 80 Plus Gold requires 87 % at 20 % load, Platinum 89 to 92 % between 20 % and 100 % load, on 115 V. The Tiers assume 87 % for the Starter’s Gold unit, 90 % for a Platinum unit, and 93 % and 91 % for the 96 GB build’s two units at their loads.
4. Apply a duty cycle. Every Tier assumes 4 hours of generation and 20 hours of idle a day. That is a guess for a personal box, and the input that moves the answer most.
5. Scale it. kWh a day × 30.4 for a month, × 365 for a year, then multiply by your own tariff. This site quotes no prices.

The Tiers’ results:

Each row is the Tier’s own rounding. The 48 GB idle, for one, is 2 × 16 W + 70 W = 102 W, which the Tier rounds to about 100 W before dividing by 0.9, giving about 111 W. The 96 GB load is the upper bound, with all four cards busy at once; a single llama.cpp stream works the cards in turn and draws less. The Starter load is sahibgasimov’s whole-machine figure with GPU and CPU at maximum, which his page doesn’t say was measured at the wall; the Tier treats it as a wall figure and an upper bound.

To redo it for your hours, keep the load and idle figures and change the split. The 48 GB build at 8 hours of generation and 16 idle is 8 h × 0.944 kW ≈ 7.6 kWh plus 16 h × 0.111 kW ≈ 1.8 kWh, about 9.3 kWh a day instead of 6.0. Held at full load around the clock it would be about 23 kWh. The 48 GB Tier makes the same comparison with jonready’s measured 1.04 kW peak at the wall: held all day, about 25 kWh, about four times its estimate.

Idle draw, and why it is the weakest number

On the Tiers’ own duty cycle, idle is a large share of the day: about 1.4 of the Starter’s 2.5 kWh, 1.9 of the 24 GB build’s 4.4, 2.2 of the 48 GB build’s 6.0, 3.6 of the 96 GB build’s 9.9. It is also the least certain input.

• Low-load efficiency. At idle the PSU runs well under 20 % of its rating, where 80 Plus sets no Platinum figure, so the real wall draw is probably somewhat higher than the Tiers’ estimates. Each Tier says so.
• The card’s own idle. Igor’s Lab measured a desktop card at idle. A headless Linux box can behave differently, and owners on NVIDIA’s developer forum report a wide spread for the RTX 3090. One saw about 25 W at idle on a headless Ubuntu server, which fell to 13 W after a monitor was plugged in once, and went back up after a reboot. Another saw 30 W headless and 20 W with HDMI connected, and reports the card stuck at 30 W once llama.cpp or Ollama had run. A third ran two used 3090s, both in P8 with persistence mode on and no display: one idled at about 7 W, the other at 76 to 78 W, and the high draw followed the card when the two were swapped. None of this is a controlled test.
• What that does to the day. If each of the 48 GB build’s two cards idles at 30 W instead of 16 W, that adds 28 W at the PSU, about 31 W at the wall at 90 %, so about 0.6 kWh over 20 idle hours. One card idling at 78 W adds about 1.4 kWh a day on its own. Measure yours: nvidia-smi --query-gpu=power.draw,pstate --format=csv -l 60 prints each card’s draw and performance state every minute, and a plug-in wall meter gives the whole-machine figure the Tiers can only estimate.

Persistence mode. NVIDIA’s documentation says that with persistence mode on, the driver stays loaded even when no client is using the GPU, which minimises the driver load latency for CUDA programs. Without it, the Linux driver unloads the GPU state each time the last client exits. NVIDIA calls the kernel setting (nvidia-smi -pm 1) legacy and recommends its persistence daemon, nvidia-persistenced; nvidia-smi uses the daemon when it is running. It matters for the power limit too: per the same documentation, the power limit goes back to its default after a driver unload, so the Tiers turn persistence on in the same boot-time service that sets the limit (the headless setup guide covers both). We have no measurement of what persistence mode alone does to idle draw on these cards; the 96 GB build expects a headless box with a model held in VRAM to idle above Igor’s Lab’s desktop figure, and the forum reports above point the same way.

The cheapest saving is not idling at all. If the box serves only you, shutting it down or suspending it when you’re done takes the idle term out of the sum, which on the Starter build is more than half the day’s energy.

The power limit is the main lever

A power limit lowers the load term directly. nvidia-smi -pl 300 sets a card’s limit in watts, within the minimum and maximum nvidia-smi -q reports; it needs root and resets on reboot or driver reload, so the Tiers reapply it at boot as Puget Systems does with a systemd service.

What it costs in speed is measured only for other work. Puget Systems found four RTX 3090s at 280 W reached over 95 % of their maximum performance, and at 250 W about 93 %, training ResNet50 in TensorFlow. That is a training benchmark, not LLM inference, and we have no inference measurement; treat it as a sign that the first 50 to 100 W come off cheaply, not as a number for your model.

What it saves, on the Tiers’ arithmetic:

• 96 GB build: at 250 W per card instead of 300 W, the load falls to about 1.35 kW at the wall and the day from about 9.9 to about 9.1 kWh, per the Tier.
• 48 GB build: two cards at 250 W give 2 × 250 W + 250 W = 750 W from the PSU, about 833 W at the wall at 90 %. Over 4 hours that is about 3.3 kWh instead of 3.8, so about 0.4 kWh a day less. This is our arithmetic on the Tier’s method, not a figure the Tier states.

The saving grows with the hours under load: a box batch-processing for 12 hours saves three times as much. A limit also lowers heat and fan speed, which is the noise lever below, and it is what keeps the multi-GPU Tiers inside their PSU and circuit budgets, which the PSU sizing guide covers. It does not cap millisecond transients.

Noise and heat

We have no noise measurement for any Tier or any Basis build. What the sources do say:

• Every watt ends up as heat in the room. At 944 W the 48 GB build heats a room like a small space heater for as long as it generates. jbarrow, whose four 3090 Tis share a room with him, writes that under full load the room “can get quite warm”, and runs bulk jobs overnight. timroy, behind one of the 24 GB build’s Basis machines, keeps his in the basement.
• The GPUs are the loud part. Stacked cards are worse than spaced ones: hav4ik names closely stacked 30-series cards blowing into each other as the reason they overheat, and moved to an open frame. skeptric’s single 3090 stayed under about 75 °C in a benchmark in a case with four extra 120 mm fans. Power-limiting lowers heat and so fan speed.
• Case or open frame. A damped case such as the Define 7 XL on the 24 GB and 48 GB builds holds noise in but also holds heat in, so fill its fan mounts. An open frame like the 96 GB build’s spaces the cards out, which helps them run cooler, but has no damping at all. Put a frame build in a room you don’t sit in.
• The PSU at idle. Corsair lists Zero RPM fan mode on the HX1200i and the AX1600i, which stops the PSU fan at low to medium loads. On a box that idles 20 hours a day, the PSU is likely silent most of the time; we have no measurement of where the fan starts.
• Coil whine is card-specific. zhonguncle, behind one of the Starter build’s Basis machines, returned an RTX 2080 Ti for it before settling on a 3060. Listen to a used card under load while you can still return it; the used RTX 3090 checklist covers that test.

If noise matters, the order of levers is: where the box sits, the power limit, then the case. Parts choice comes last.

Related

• Starter 12 GB build
• 24 GB build
• 48 GB build
• 96 GB build
• PSU
• Case/Frame

Last reviewed 2026-11-12