Navigating Cost Efficient Hardware in these Volatile Times

Where to even begin on this one...I guess I should start by acknowledging the risk vs reward for vendors other than nvidia, so:

Yes I understand that nvidia are dominant currently on speed (llm and imagegen) and software ecosystem. I am too am hopefuly that software stack support continues to improve with other vendors. The current lag for other vendors is not a priority concern (it falls behind the primary price concern).

Entry points for "decent" local inferencing look to be circa AUD 2000-2500+ (the price of a 2nd hand 3090, or two 3060s, b60 48gb, r9700 32gb), and yes other older architectures are available (eg. V100)...but they really end up around the same costs once all said and done.

Workload will be a mixed bag with some DL/ML training/development projects, but when not doing that I'll consume HF models to run a coding agent, imagegen (just for the fun of it/try out video and for laughs), and probably dive into finetune/distilling.

Hence, I'm looking around that sub AUD 5k mark to dive in and FAFO, but I don't want to be needlessly cavalier in my purchase either...

Asking AI is no real use because it's out of touch with modern markets until you correct it a bunch. It's also out of touch with software stack development/progress.

So I put it to the hive mind, where is the money best spent for diving deeper into local?

- accepting prices wont change and pay 2k a piece for 2nd hand 3090's.

- find some 16gb variants and get 4 instead of 2.

- dive into the intel arc rabbit hole with the b60 dual (48gb, but it's just 2xgpu on a single pci slot)

- AMD path (r9700 seems the best price point but could wait 3 months to see what the new 10x series looks like)

- unified memory systems (honestly the price vs performance just doesn't seem worth it at this point)

ETA: I have a threadripper and a lot of DDR4 RAM, but current motherboard is constrained to 2 x16 physical slots. I also have a nvidia gpu already....but I dont want that to impact the core of the discussion as I could move that into a different system and use it to server models that fit wholly in its vram footprint.