Are you ready for Le Chaton FAT or still wasting money on GPUs?

According to rumors (spread by myself) Le Chaton FAT will be 26T-a3b and I AM READY for it.

Let's be real, I can't afford that many 5060Ti, so I got 12x Gen 4 3.2 TB (two per card). This gives me about 60GBs bandwidth on 30TB.

Added 256gb ddr4 just for kv cache, but I can also write KV-cache to the disks, these are high endurance drives.

Are you ready for the next era of local inference?

Jokes aside, this is what I use for my HF_HOME - model and dataset storage. I'm also setting up a few containers, but it's not running any heavy compute stuff, the CPU is only a 3945WX (12c/24t).

The pool is actually raidz2, so I avoid all that worry of having agents delete stuff. I just zfs snapshot and no rm -rf foo-bar has me sweat.