Welcome to 2014 - my new rig

Hi all,

Probably for some others me, there is a more conservative budget when it comes to their AI hobby. I've been collecting basically e-waste and could now assemble something partially working from them.

Specs:

$ neofetch .-/+oossssoo+\-. molbal@... ´:+ssssssssssssssssss+:` ------------- -+ssssssssssssssssssyyssss+- OS: Ubuntu 26.04 LTS x86_64 .ossssssssssssssssssdMMMNysssso. Host: Supermicro X9DR3-F /ssssssssssshdmmNNmmyNMMMMhssssss\ Kernel: 7.0.0-27-generic +ssssssssshmydMMMMMMMNddddyssssssss+ Uptime: 3 hours, 15 mins /sssssssshNMMMyhhyyyyhmNMMMNhssssssss\ Packages: 1132 (dpkg), 3 (snap) .ssssssssdMMMNhsssssssssshNMMMdssssssss. Shell: bash 5.3.9 +sssshhhyNMMNyssssssssssssyNMMMysssssss+ Resolution: 1024x768 ossyNMMMNyMMhsssssssssssssshmmmhssssssso Terminal: /dev/pts/3 ossyNMMMNyMMhsssssssssssssshmmmhssssssso CPU: Intel Xeon E5-2660 v2 (40) @ 3.000GHz +sssshhhyNMMNyssssssssssssyNMMMysssssss+ GPU: NVIDIA GeForce RTX 3090 .ssssssssdMMMNhsssssssssshNMMMdssssssss. Memory: 147173MiB / 189019MiB \sssssssshNMMMyhhyyyyhdNMMMNhssssssss/ +sssssssssdmydMMMMMMMMddddyssssssss+ \ssssssssssshdmNNNNmyNMMMMhssssss/ .ossssssssssssssssssdMMMNysssso. -+sssssssssssssssssyyyssss+- `:+ssssssssssssssssss+:` .-\+oossssoo+/-. 

And the prices of all. (Location Netherlands)

Component	Model	Price	Note
Motherboard	Supermicro X9DR3-F	70€	Bought refurbished, only 6 out of 16 memory slots work
Memory	8x32GB DDR3 1866MHz	160€	Bought used from the local marketplace. 2 currently sitting outside the computer
GPU	RTX 3090	400€	Bought for this cheap, because it had faulty cooling. Deshrouded it and zip-tied 3x90mm static pressure fans. Works fine now
CPU	2x Xeon E5-2660	2x20€	These are 10 core, 20 thread, 95W CPUs. Unfortunately quite old
Storage	512GB SATA SSD	30€	I put in 30€ because that's what it would cost, but I had it laying around
PSU	Corsair RM1000x	95€

In theory, this combination would get me 2x4 channels of DDR3 1866MHz, which is still not fast, but at least it would be 8 channels. Instead of the original 8 channel 256GB I work with ??? channel 192GB RAM.

In the journey to arrive at this beauty, I first had an eGPU dock to hook up that GPU to my laptop, which made me realize my laptop is USB4-compliant in theory, but not in practice, so that eGPU dock went back to the seller. Then I ordered a Fujistu Celsius server, which was destroyed by the courier company (DPD🖕), then I ordered a motherboard from Ali Express, which also arrived broken, then I ordered a replacement one, which was fine, but then my PSU was defective so I sent that back, then I ordered this supermicro board and that seems at least partially working. So while it is cheap, at least compared to 2026 computer part prices, I paid for it with sanity and patience.

So for now it has the memory channels all jumbled up because of the 6 working, but I am happy that at least it booted, so I ran some tests on it.

Deepseek V4 Flash, Unsloth's Q4_K_XL quant

The setup got me 8k context size only, and the pp speed was slower than I expected (37.47t/s) and tg was in line with my expectations (6.62t/s)

This is the llama-server config for it:

[deepseek-v4-flash] model = /workspace/storage-tmp/UD-Q4_K_XL/DeepSeek-V4-Flash-UD-Q4_K_XL-00001-of-00005.gguf load-on-startup = false ctx-size = 32768 temp = 1.0 top-p = 1.0 chat-template-kwargs = {"reasoning_effort":"high"} ; --- Smart Offloading --- ; Let auto fill the 3090, but explicitly force all routed MoE experts to stay on the CPU ngl = auto cpu-moe = true ; --- Dual-Socket NUMA Tuning --- ; Distribute workload across both Xeons and disable mmap to prevent Node 0 memory crowding numa = distribute mmap = false 

Cydonia v4.3

This is a Mistral Small 3.2 finetune. It barely offloads, so that is a usual thing. 'Auto' sets it at 16k context. It does 308.92 t/s pp and 7.8 t/s tg, which is slower than what I would expect.

This is the llama-server config for it.

[cydonia] model = /workspace/storage-tmp/llama-models/Cydonia-24B-4.3-Q5_K_M.gguf mmproj = /workspace/storage-tmp/llama-models/Mistral-Small-3.2-24B-Instruct-2506-mmproj-F32.gguf load-on-startup = false ctx-size = 65536 ngl = auto temp = 0.15 

https://preview.redd.it/zp0z94ichsdh1.jpg?width=1848&format=pjpg&auto=webp&s=12ad4e48dbdbb53f765c959529cf9a2c721a9e10

If you are interested I can test some other things. It having an RTX 3090, it naturally works very well with diffusion models in INT8 ConvRot format (Krea2 and Flux2 Klein namely)