Ryzen 7000 Testing

[agent2]

Officially announced…

AMD unveils Radeon Pro W7900 48GB and Pro W7800 32GB RDNA3 workstation GPUs - VideoCardz.com

AMD Radeon Pro W7900 & W7800 announced The company confirms specs and pricing of Radeon PRO GPUs with RDNA3 architecture. AMD Radeon Pro W7900 features 96 Compute Units (full Navi 31) combined with 48GB of GDDR6 ECC memory. This is a flagship workstation solution based on the same GPU as the...

videocardz.com

Well now that we know for sure, the ball is in Apple's court to not mess around. I also have high hopes that this will finally mean RDNA3 support in macOS. As long as that's not the case, my 6900XT will continue to serve me well. I'm really happy AMD decided to stick it to the RTX6000 with the FULL Navi 31 die and juicy 48GBs of VRAM. That thing will be a hell of a beast with three slots, it would massacre gaming for sure. And with ROCm support in Windows incoming, this is looking very nice indeed!

 

[CaseySJ]

I'm beginning to see the power of TrueNAS, particularly after adding TrueCharts catalog:



Disk Speed Test with 10GbE connection between Hackintosh and TrueNAS:




LibreOffice and Pi-hole running over the network:


Gigabyte AQC-113C Ethernet (Aquantia 10GbE) and PCIe x1 4-port SATA expander


P.S. Completed an in-place upgrade to 22.12.2.

 

[DarkSilentSC]

CaseySJ - What filesystem are you using for your TrueNAS? If ZFS, you think ECC Memory is not critical?

 

[CaseySJ]

CaseySJ - What filesystem are you using for your TrueNAS? If ZFS, you think ECC Memory is not critical?

TrueNAS uses ZFS exclusively, but ECC memory is not strictly necessary. If we intend to run the NAS continuously for months then ECC is recommended. There’s a fair bit of debate on this topic!

 

[etorix]

CaseySJ - What filesystem are you using for your TrueNAS? If ZFS, you think ECC Memory is not critical?

TrueNAS only supports ZFS as file system.
As for ECC memory, pages have been written about it… Fact is ZFS is designed with the assumption of ECC memory; ZFS checksums everything on disk to prevent bit rot but trusts what's in memory, relying on ECC to catch bit flips (there's a debug flag to force ZFS to cheksums RAM as well). So without ECC there's a possibility that ZFS could serve incorrect data from RAM, and there's the remote possibility that a bit flip in a stored checksum causes ZFS to actually corrupts data to match the checksum—a catastrophic scenario known as "the scrub of death". This remains relatively unlikely, and ZFS without ECC is arguably still more secure than just about any other file system (HFS, AFS, NTFS, ext2/3/4, ufs, xfs…) even with ECC.

But ZFS is a nice fit for paranoids, and if you're paranoid about your data it makes no sense not to use ECC RAM when deploying ZFS.

My personal opinion and advice is that, if you're recycling an old desktop build into a NAS (possibly 10 GbE or faster…) it is acceptable, as a first intention, to use what you have—which is likely not ECC. But if you're actually buying a motherboard and/or CPU to build your NAS, you should do it by the textbook and go for ECC RAM.
ECC benefits any server—actually any computer—so the advice applies to any kind of NAS, not only ZFS solutions TrueNAS or XigmaNAS. Incidentally, older motherboards are likely to have more SATA ports than newer ones, and serving files does not require extreme computing power so old gear does well—as long as one does not go all the way back to pre-Haswell "space heaters". So reuse your old builds or go for refurbished/second-hand server-grade hardware (MB, CPU, ECC RAM, NIC…) and save your money for new hard drives and a new well-sized Gold PSU or better (a NAS is mostly idle, so the high efficiency of Platinum/Titanium PSU under low load is of value—though possibly not to the point of paying back the extra cost of the highest grade).
10 GbE is old tech in server world, so second-hand server-grade NICs are cheap: Solarflare 5122F/6122F/7122F go for $50 on eBay; Chelsio T520, from $100-$150 with some patience or some effort shopping around (even the non-SO variants, which are better). These have excellent native support in Linux (OpenMediaVault, Unraid, TrueNAS SCALE) and FreeBSD (TrueNAS CORE, XigmaNAS). Bonus: These oldies have drivers for macOS and can work on the Hackintosh side as well—without kernel patches and/or requiring working AppleVTD like Aquantia or Intel X500 NICs.

(Edit: Of course, I've been ninja'd…)

 

[CaseySJ]

Aquantia AQC113 is fully supported by TrueNAS and functions at 1GbE, 2.5GbE, 5GbE and 10GbE. Many SFP+ cards support only 10GbE and 1GbE.

 

[CaseySJ]

** TrueNAS Experience Updates **

After nearly two days with TrueNAS I thought I would share some experiences and impressions in no particular order:

• When installing apps, particularly from the rich collection of apps from TrueCharts catalog, we have to configure the app's container parameters carefully, particularly App Additional Storage
• We also have to delve into the world of ACL (access control lists), which are not so bad after giving them a try
  • They allow other user accounts to get any combination of read/write/execute permissions on a dataset owned by a different user
• Configuring a Samba share (SMB) that works with Carbon Copy Cloner on macOS is a bit tricky:
  • Set Purpose to No presets
  • Set Other Options:
    • Enable Time Machine
    • Enable SMB2/SMB3 Durable Handles
    • Enable SMB2/SMB3 Apple Extensions in the SMB service section
  • Credit goes to this post
• Configuring Jellyfin (free alternative to Plex) was completely non-intuitive. Thankfully there's this detailed guide
  • Jellyfin looks very nice!
• Configuring FileZilla to directly transfer files from my QNAP unit was tricky as well, but in the end the App Additional Storage technique worked nicely
• I've added two new VDEVs:
  • Log VDEV using two mirrored SSDs
  • Cache VDEV using single SSD
• Will experiment with Deduplication later; will also add Dedup VDEV
• Apps that open using https will likely fail to appear (probably due to missing certificate), but we can replace https with http
• Also used Virtualization to install Windows 10, but Windows 11 would not install ("this computer does not meet minimum requirements")
  • Next Step: Install a spare Nvidia GT 1030 small-form-factor GPU and pass it through to Windows

The TrueNAS system is still in experimental mode (i.e. learning mode). But I don't expect to wipe and reinstall. Instead, just a little bit of dataset gymnastics may be all that's required.

I'm surprised how many of my systems are 10GbE and 2.5GbE capable. I'm now thinking of upgrading the entire home network because these devices are located in different rooms and on different floors. To this end, Ubiquity's lineup is looking very appealing because of their UniFi OS that allows full network control and administration from a single Dream Machine or self-hosted computer. I can replace my aging home security cameras as well. This will be a project for the near-to-medium term, not right now.

Update:

• I was struggling to find an rsync tool on QuTS (QNAP), which is why I resorted to FileZilla
• Just found the HBS 3 Hybrid Sync app on QuTS, which handles rsync beautifully
• Below, a job has been created to rsync the Master MP3 Library folder from QNAP to TrueNAS

 

[etorix]

I've added two new VDEVs:

• Log VDEV using two mirrored SSDs
• Cache VDEV using single SSD

SLOG is only useful for sync writes; it is not used for async writes. Async is always a lot faster than sync, even with a SLOG. If you use this NAS for hosting TimeMachine, you actually have sync workload over SMB… but disabling sync writes (sync=never) may be an acceptable solution.
SLOG does not need to be mirrored (the content of the SLOG is not pool critical and is not even read unless there was an unclean shutdown), but a SLOG device needs to have Power-Loss Protection to fulfil its duty in case of an unclean shutdown. Which means that SLOG has to be either a data centre-grade SSD with PLP (in M.2 format, these are generally 22110), Optane (M10 or consumer 900p/905p are acceptable), or a Radian RMS-200 / RMS-300 card (battery-backed RAM). Consumer SSDs are NOT suitable SLOG devices.

L2ARC (cache) requires RAM for its management. With CORE the rule of thumb is that L2ARC should not be large than 5 to 10 times RAM. With SCALE, and the Linux kernel allocator limiting ARC to 50% of RAM, I'm unsure if the ratio should still apply to RAM to to ARC, i.e. half of RAM. Too large a L2ARC actually reduces performance by evicting useful data from ARC.
In general, it is better to use the NAS for some time without L2ARC, keep an eye on ARC statistics with the arc_summary script and only add a L2ARC if the ARC hit ratio is too low. Hit ratios close to 100% will not be improved by adding a L2ARC.

• Will experiment with Deduplication later; will also add Dedup VDEV

Contrary to SLOG and L2ARC, which are not pool-critical and can always be removed, a special vdev/dedup vdev is a critical pool component and can only be removed if the pool is entirely composed of mirrors. If any raidz vdev is used, the addition of a special/dedup to a pool is irreversible.
Special/dedup requires at least the same degree of redundancy as the data vdevs on HDDs—meaning 2-way mirror for raidz1 (not recommended with large HDDs), or 3-way mirror to match (recommended) raidz2. With SLOG, L2ARC and dedup, you're at risk of running out of PCIe lanes…
L2ARC can and will hold the DDT for all read purposes, while being reversible; a dedup vdev will only improve performance with respect to writes to the pool.

 

[CaseySJ]

@etorix

Very helpful insights.

This NAS will be used for a variety of purposes:

• Carbon Copy Cloner backups (macOS)
• Acronis backups (Windows)
• Media hosting (Jellyfin)
• Backing up the QNAP NAS
• Various apps
• Other use cases yet to be discovered

The current ARC hit ratio is averaging 99% but with very limited use so far. It’s a good idea to monitor cache performance with and without L2 VDEV. Same for Log VDEV, although an Intel Optane M10 64GB version is relatively inexpensive on NewEgg.

When setting up the Log VDEV, TrueNAS did not allow me to specify a single SATA SSD. It wanted two or more. Perhaps it will allow a single Optane M10 to serve that purpose.

Since Dedup VDEV cannot be removed without destroying the pool, I may try it on a simple expendable pool.

I’ve ordered a QNAP QM2-2P10G1TB that combines an Aquantia AQC113C and two NVMe x4 M.2 slots. It’s compatible with TrueNAS. This will free up the PCIe x4 slot that currently houses the standalone AQC113C. Into that slot will go the Nvidia GT 1030, which is electrically x4.

 

[etorix]

Very helpful insights.

You're welcome… as long as no moderator complain that we're drifting far away from hackintoshing AM5.

This NAS will be used for a variety of purposes:

• Carbon Copy Cloner backups (macOS)
• Acronis backups (Windows)
• Media hosting (Jellyfin)
• Backing up the QNAP NAS

None of that should generate sync writes. SMB is generally asynchronous; only TimeMachine does generate sync requests over SMB.
So based on that you do not need a SLOG at all. Which is the usual case for home users; sync writes are important for database applications and business-critical VMs, which are hardly home-use cases.
(I cannot stress highly enough that SLOG is not even used for async writes.)

The current ARC hit ratio is averaging 99% but with very limited use so far. It’s a good idea to monitor cache performance with and without L2 VDEV.

Meaningful readings are "after some time" (possibly several days), to ensure that ARC is properly filled. If it stays around 99% you do not need a L2ARC. Which, again, is a very common case if there's enough RAM.

Same for Log VDEV, although an Intel Optane M10 64GB version is relatively inexpensive on NewEgg.

Requirements for a proper SLOG are PLP, low write latency and high endurance. As for capacity, the SLOG only needs to hold up to two transaction groups, i.e. ten seconds worth of transactions with default values, as this is the maximal amount that ZFS will cache before committing everything to disk (stalling further operations until it's done if the pool cannot cope with the throughput). So only about 10 GB is necessary for a 10 GbE NAS, but the 64 GB M10 Optane would be a good choice because the M10 entry-level line of Optane is actually not that good with respect to write throughput and endurance, and increasing capacity does help with both.
Again, I do not want to prevent you from hoarding a few 64 GB M10 "just in case" if you have the opportunity (I recently did the same with a few second-hand 100 GB P4801X), but as things stand it seems that you have no use at all for a SLOG.

When setting up the Log VDEV, TrueNAS did not allow me to specify a single SATA SSD. It wanted two or more. Perhaps it will allow a single Optane M10 to serve that purpose.

Curious… I had no trouble setting up a single drive as log vdev in CORE.

I’ve ordered a QNAP QM2-2P10G1TB that combines an Aquantia AQC113C and two NVMe x4 M.2 slots.

QNAP keeps the actual specifications close to its chest, but the ACQ113C is a PCIe 4.0x4 chip so to host the NIC and two PCIe 3.0x4 M.2 drives on a PCIe 3.0x8 card there must be a PLX switch under that big copper cooler—maybe a 24-lane PLX 8724 or 8725. The PLX is just transparent to the OS so, sure, it works, and $219 pays for both the PLX and the AQC113C chips. But I cannot help to think that this is a lot of money thrown at solving a problem of lack of space or lack of PCIe lanes which is self-inflicted, and may not exist in the first place. So far I see no use for SLOG, and no proven use case for a L2ARC yet. If you can resist the itch to experiment with resource-hog dedup, or at least handle dedup with a single device as persistent L2ARC rather than with multiple drives as dedup vdev, the board should have enough lanes without throwing in an exotic PLX-based card.

 

[CaseySJ]

Although talk of TtueNAS has drifted from the intent of this thread, it has not drifted from the intent of the Chit Chat forum, which is, "Chat about almost anything as long as it is within the rules."

My intent is to keep tying this discussion back to Hackintosh, such as configuring TrueNAS for Carbon Copy Cloner (sparse disk bundles). The original intent was to use an AM5 Ryzen 5 7600. Still scratching my head over why that failed.

Some comments:

• The QNAP QM2 card is for maximizing the use of available slots. The extra NVMe ports can come in handy later for Proxmox (I don’t intend to deploy TrueNAS in a VM, but I do want to test it in a VM), or for non-SSD M.2 devices
• AQC-113C won't saturate a PCIe 3.0 bus, so the QM2 being PCIe 3.0 is okay; I was using AQC-113C and AQC-107 in a x1 slot on the Asus X670E Gene when investigating the AppleVTD kext patch
• Because I’m building this NAS as an enthusiast, the specs and configuration are not being driven exclusively by need. I do have a need for the large storage space, but most other hardware choices are being driven by “what’s cool” while still being price-practical. However, it is still very helpful to know what some of these ZFS capabilities are and are not suitable for

 

[CaseySJ]

** Testing the 1920x512 LCD Panel **

After receiving a mini-HDMI cable from Amazon, I was finally able to test the 12.6" LCD panel. It's working well, but I'll have to McGuyver a way to install this. But first, some photos. Will connect the touch panel cables soon.

Asus BIOS (left) and Grub boot options (right):


TrueNAS Scale boot progress:


Boot complete; IP address shown (left) and dry-fit of components (right)


There are no instructions, just pictures. But stock photos are good enough...

 

[amdeez]

Excellent observation, depends on the fix pat you use

I changed fix pat to Shane's and got huge gains!
I am running these tests on 12.6.5 powering a 2160p (120hz) and 1440p (120hz) at high refresh rates so the OpenCL scores look much closer to what I was expecting!
GB6 OPENCL: 126129 vs 22505
GB5 OPENCL: 132182 vs 85619

 

[CaseySJ]

@leesurone,

That was a good suggestion: Power on/off issue is due to FLEX power supply. Testing with an EVGA 850W SuperNova solves this problem, but other issues remain.

PSU is being returned to Amazon, but being replaced by a more expensive unit also from Amazon.

I've ordered this 500W Flex PSU from Amazon, which is expected tomorrow:

https://www.amazon.com/dp/B08BF1H5R1

Long overdue update on the replacement Flex PSU from FSP:

• Thankfully it does not suffer from the reboot problem!
• Whisper quiet fan
• Working very well for past several days

 

[Galve2000]

I changed fix pat to Shane's and got huge gains!

Which patch are you using?

 

[CaseySJ]

Which patch are you using?

This one:

 

[Galve2000]

This one:

very nice,

is this for Ventura only? what happens if I use it in Monterey?

 

[amdeez]

very nice,

is this for Ventura only? what happens if I use it in Monterey?

works in Monterey, that's what my screenshots were from

 

[AMDPUSH]

I changed fix pat to Shane's and got huge gains!
I am running these tests on 12.6.5 powering a 2160p (120hz) and 1440p (120hz) at high refresh rates so the OpenCL scores look much closer to what I was expecting!
GB6 OPENCL: 126129 vs 22505
GB5 OPENCL: 132182 vs 85619

ok, and now could make the test with it for Metal, then we can have an overall view of the graphic performances

 

[Galve2000]

works in Monterey, that's what my screenshots were from

Here's what the Kernel section of my Config.plist looks like:



I think I am already using the appropriate patch. although I do wonder what will happen if I try the "algray" patch instead.

in other news, I just bought a 6950XT and I think I am going to install it tonight to finally start doing some proper Metal scores. the only hickup I see is that I don't know where the 3rd 8-pin connector cable is being stored. LOLOLOL. oops.

I need to scroll up to where this post where @ExtremeXT mentions that Carswell is the framebuffer for the 6900XT.

edited to add:

so I did scroll up, and I note that the spoofed device ID "BF73000" is the same for both the 6900XT and the 6650XT.

can this really be true?

if I leave the frame buffer to Henbury instead of Carswell for the 6950XT will this be a very bad thing?

fingers crossed everyone.