Storage for model weights

A model drive does one job: hold the files you might run and hand one over quickly when you start the server. That makes it the least demanding part of a local-AI box. Capacity decides how many models you keep. Read speed decides how long a cold start takes, and only a cold start. Endurance barely matters. Each point below is worked from the GGUF files the Build Tiers run and the drives in the Catalog.

How much space a library takes

A GGUF file is the size of its listing on Hugging Face, and the Tiers’ Model fit rows give each one:

The rows add up to 343.91 GB, but the 48 and 96 GB builds share the 42.52 GB Llama 3.3 70B Q4_K_M file, so every distinct GGUF file named on the four Tiers comes to about 301 GB. A 2 TB drive such as the Samsung 990 PRO on the 24, 48 and 96 GB builds holds that set more than six times over.

Libraries grow anyway, mostly sideways: several quantisations of the same model to compare, a new release every few weeks, and the ones you never delete. Two 70B files, Q4_K_M and Q8_0, are already 117.5 GB. If you expect to keep a shelf of 70B-class and larger models, the 4 TB WD_BLACK SN850X, the Tiers’ alternate, doubles the room at the same Gen4 speed. Drive capacities are decimal, like file listings, so the numbers compare directly; the file system takes a little off the top.

Load time is bounded by read speed

On a cold start the whole file comes off the drive once, so the drive’s sequential read sets a lower bound on load time: file size ÷ read speed. The Catalog’s figures:

These are floors, not measurements. A real load also parses the file, allocates memory and copies the weights to the GPUs, and we have no measured load time for any drive here. The copy to the cards is not the limit: per the PCIe lanes and risers guide, a Gen4 x8 link moves about 15.75 GB/s, twice what a Gen4 drive reads, and the drive’s own x4 link caps a Gen4 drive at about 7.9 GB/s.

After the first load, RAM serves the file

llama.cpp memory-maps the model by default, so the file is read through the Linux page cache, which keeps file data in memory to avoid disk reads on later accesses. Stop the server and start it again, or switch back to a model you used earlier, and the file comes from RAM instead of the drive, as long as nothing has pushed it out. On the 48 GB build, 128 GB of RAM keeps the 42.52 GB Q4_K_M file cached with plenty to spare; on the 96 GB build, 256 GB keeps all three of its model files cached at once. How much RAM that takes, and what happens when layers are offloaded to RAM, is in the system RAM guide.

That is also why the drive matters so little day to day: a box that runs one model for weeks reads it from disk once.

Where the weights should live

• One fast NVMe drive for everything is fine for a single box. That is how the 24, 48 and 96 GB builds are listed: one drive for the OS and the models.
• A second drive for the library keeps the OS, its logs and updates apart from the model files, so a full library can’t fill the system drive, and you can reinstall the OS without downloading the models again. It is a convenience, not a speed upgrade.
• Slower storage only slows the first load. A SATA SSD such as Samsung’s 870 EVO is rated at 560 MB/s read, which puts the floor at about 76 s for the 42.52 GB file and about 134 s (2.2 minutes) for the 74.98 GB one, against about 6 and 10 s on the 990 PRO. A hard drive or a NAS is slower again; we have no measurement of either. After that first load, the page cache serves repeats from RAM if the file fits, whatever it came from.

Filling the library is usually limited by your internet connection, not the drive: even a 1 Gbit/s connection delivers at most 125 MB/s, below the SATA drive’s rated speed.

Why PCIe 5 buys little here

A Gen5 drive halves the cold-start floor: about 5 seconds saved on the 74.98 GB file, about 3 on the 42.52 GB one. That is the whole benefit. Generation speed doesn’t change, because once the model is loaded the drive is idle, and warm restarts come from the page cache either way. Two more limits:

• It needs a Gen5 slot. On the X870E Taichi, only M2_1 runs PCIe Gen5 x4; M2_2 to M2_4 are Gen4 x4 (manual PDF page 8). The ROMED8-2T on the 96 GB build has two M.2 slots, both PCIe 4.0 x4 (manual PDF page 9), so a Gen5 drive there reads at Gen4 speed. M2_1 also shares lanes with PCIE2: with the PE8_SEL and PE16_SEL jumpers at their default, which keeps PCIE2 at x16, M2_1 is disabled (manual PDF pages 9 and 33). Use M2_2, as the 96 GB build does.
• Heat. The Catalog notes that the T705 and the 9100 PRO in their bare versions want a heatsink over the slot for full performance, which matters more in a closed case next to hot GPUs.

Spend the difference on capacity, or on RAM, which makes repeat loads faster than any drive.

Endurance is a weak concern

Endurance ratings count writes, and model weights are almost all reads. Samsung rates the 2 TB 990 PRO at 1,200 TBW, and the 4 TB SN850X is listed at 2,400. Downloading every GGUF file on the Tiers, about 301 GB, uses about 0.025 % of the 990 PRO’s rating. Re-downloading the 42.52 GB 70B file every day for five years would write about 78 TB, about 6.5 % of it. Loading and running a model only reads the file, and reads don’t count against a TBW rating, which counts terabytes written.

What does write is everything else on the same drive: the operating system, logs, swap if the box runs short of RAM, and any fine-tuning checkpoints, which this site doesn’t size. For an inference box, buy on capacity and treat endurance as a tiebreaker.

Parts in this guide

Store links may earn us a commission at no cost to you. As an Amazon Associate we earn from qualifying purchases.

Related

• Storage
• Samsung 990 PRO 2TB
• WD_BLACK SN850X 4TB
• Crucial T500 2TB
• Samsung 9100 PRO 4TB
• Crucial T705 2TB
• 24 GB build
• 48 GB build
• 96 GB build

Last reviewed 2026-11-12