I came so damn close!
CPU: AMD Ryzen 3950X
Motherboard: Asus ROG Strix X570-F
SSD: 860 Evo 1TB (Sata)
Memory: 2x16GB DDR4 3600 CAS 19 RAM (F4-3600C19D-32GVRB)
In VESA mode (no graphics acceleration), everything worked perfectly—or as perfectly as a system without graphics acceleration can ever work. I didn't do anything overly special to get the machine to boot this way. I used this kernel and configured Clover to use a Mac Pro 3,1 SMBIOS. AppleIGB.kext allowed my motherboard's native Ethernet to work, and I used a Inateck KT4004 PCIe card for USB alongside a USB DAC for audio. I installed Firefox and iWork 09 and spent several hours using my computer normally, without issue. I was also able to VNC into the machine once I enabled Screen Sharing.
Graphics acceleration is where everything fell apart. I'd bought two different graphics cards for this project: a GTX 780 6 GB and a GTX 780 Ti. The former is supposed to be natively supported by Mavericks; the latter is supported only when the Web Drivers are installed.
And when I booted with the GTX 780, graphics acceleration did work! I could see my desktop in full resolution, and all of OS X's GPU-only graphical effects, such as a transparent top menu bar, were visible. Unfortunately, the entire system was terribly, horribly slow—attempting to open a new Finder window resulted in a beach ball that frequently lasted for 60 seconds or more, as did opening any new folders. Other programs were similarly slow, to such a degree that the machine was completely useable. When programs did open, they would crash frequently.
The following message was always visible in the console following a freeze:
kernel: NVDA(Private): Channel timeout!
Installing the nVidia Web Drivers resulted in a different problem—the machine would boot to a gray screen with my mouse cursor, and stay there. I could move my mouse around the screen, but not do anything else. Attempting to VNC into the machine caused it to freeze (such that I could no longer move my mouse). Switching between my 780 and my 780 Ti had no effect.
Booting my 780 Ti without the web drivers installed resulted in no video output (which was not particularly surprising, as this card requires Web Drivers in Mavericks). Interestingly, however, VNC'ing into the machine seemingly caused it to freeze, like when the Web Drivers were installed.
I spent many hours messing with different configurations and boot arguments (and I asked for help in the Discord), but I eventually decided it was time to give up. The one thing I would have liked to try is using an AMD GPU, but at this point it seemed exceedingly unlikely to work.
My current plan is to build a new computer around an Intel i7 4790K (a Haswell part). But if anyone is able to get further than I did with Mavericks on modern hardware, please document your progress. I'm not expecting my obsession with Mavericks to wane anytime soon, so it would be nice to return to this some day.