I have one 4-bay NAS that is many years old and two 4-bay direct-attached storage arrays. That’s 12 drives of assorted sizes.
As you probably know, ZFS is not too good at accommodating that. Drives in the same vdev are best identical, or of very similar size (it's possible to have different vdevs with drives of different sizes tough, there will be a performance penalty as the vdev with more free space will get more writes, but no space penalty).
The plan is to replace all 3 units with this 8-bay system running TrueNAS with ZFS.
"ZFS is not a backup", so keeping another NAS (off-site and/or off-line) to back up the first is recommended.
I want to enable deduplication, hence the extreme amount of memory.
Ouch! Dedup is best avoided, really…
64 GB RAM is not even "extreme". There's a
recommended minimum of 16 GB, scaling up with storage space (rule of thumb: 1 GB RAM per 1 TB data for a start, relaxing at some point). For making full use of a 10 GbE link, make that
32 GB to have a larger ARC and enough space to hold the write cache (= 2 transaction groups; by default, that would be 10 seconds worth of data @ 10 Gb/s, so up to 12 GB). In this light, 64 GB is "just" a reasonable size to serve a 100+ TB pool over 10 GbE.
Extra services (jails, VMs, dedup…) come with their requirements
on top of that. For dedup, the rule of thumb is an extra 5 GB per TB of deduped data to hold the dedup table (DDT, like its namesake it is a rather toxic affair…).
There's little point in dedup'ing only a few TB—which is all that 64 GB would support.
If this is mostly static data, one may get away with dedup and only 64 GB by throwing in a 280 GB Optane 900p drive to serve as persistent L2ARC (possibly "metadata-only" L2ARC) to hold the DDT. But the best way is to
avoid dedup and just rely on built-in compression (lz4 or zstd) to do its job. 64 GB is, coincidentally, the recommended minimum to consider a L2ARC.
The TrueNAS forums occasionally have people who come across ZFS de-duplicate, and want to investigate its use. Or think it is a good idea, and want to implement it. Here are some suggested configuration details: Understand that you need CPU...
www.truenas.com
All Zen 4 CPUs have built-in iGPU, so plan is to connect one video output to the touch screen.
I suppose that display would be fed from a VM since TrueNAS only outputs a rather boring administrative console…
The mATX (not mini-ITX) has 6 SATA ports so a PCIe X1 card is arriving soon with 4 additional ports.
Re-ouch! That SATA card is likely a liability. ZFS really wants to deal with the SATA controller built into the chipset or with a SAS HBA (LSI 2008/2308/3008—no RAID controller), and to
own the controller in either case, but nothing else.
1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...
www.truenas.com
Any option to reconsider part of the build at this stage and go for either:
- An AsRockRack B650D4U motherboard (8 SATA and likely the best possible support for ECC RAM with Ryzen);
- A C236/C246 Intel motherboard with matching Core i3 (for serving files to Macs through SMB, which is single threaded, few cores at high clock is enough);
- A (refurbished) Xeon E5v3/4 system (again, a low core part is enough); or
- A (refurbished) Xeon D-1500 or (new) Atom C3000 system (lower clocks for SMB… but terrific low power NAS platform otherwise)?
The last two options take RDIMM, which is a great way to fill up a system with RAM for cheap.
The MSI motherboard has 3 slots:
- X16 which I will leave empty for now
- X1 for a 4-port SATA expander
- X4 (in a long x16 slot) that will accommodate an extra Gigabyte AQC113C 10GbE card that has been sitting on the shelf for several months
Driver support for AQC NICs is TrueNAS is, at best, dubious. Solarflare NICs have good drivers, and a PCIe 3.0 SFN7122FF card costs $50 on eBay.
With a LSI 9200 (sufficient for HDDs) or 9300 (for SATA SSDs) in the other long slot, there are just enough slots. The x1 slot is essentially useless in a server.
Now I know who to turn to for questions!
Don't forget the TrueNAS forum
The TrueNAS Community is your one-stop shop for discussion and troubleshooting of NAS hardware and open source software products like FreeNAS, TrueNAS, and TrueCommand.
www.truenas.com