r/LocalLLaMA · · 2 min read

A tool I built to generate 3D objects with functional, articulated parts. It's on github, and is mostly LLM-agnostic.

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

A tool I built to generate 3D objects with functional, articulated parts. It's on github, and is mostly LLM-agnostic.

The video shows how my pipeline produces a 3D washing machine, composed of separated, functional parts (instead of monolithic 3D blobs). There's also hinge/socket articulation, so the internal assembly actually churns or rotates.

The problem I'm solving: pretty much every text-to-3D pipeline right now is just diffusion (or diffusion-like) weights producing mesh blobs. Want to swap the scope on a generated gun? Change one word in your prompt, the whole or most of the thing regenerates from scratch, because the model has zero concept that a gun has parts. It's all undifferentiated point clouds to these systems.

I tried to change that. My pipeline uses an LLM as a structured code compiler, instead of an image generator. It writes native Blender Python code blocks that target specific nodes in the scene graph.

The trick is that everything compiles through Blender's actual scene graph structures instead of pixel or point-cloud diffusion. Final export is a clean multi-part GLB with transform nodes and working pivot axes preserved.

Stack:
Frontend is Flutter with a Three.js viewport harness for in-browser rendering and node manipulation. By default it hits my hosted API, but I made it model-agnostic for self-hosting.

Frontend repo is open source: https://github.com/RareSense/Nova3D

Also some bad news: local models are getting there, but they still hallucinate Blender's internal matrix math functions pretty badly on complex transforms. I spent way too long debugging "correct-looking" code that was actually rotating things into the fourth dimension. The pipeline code itself is fully LLM-agnostic, byt for best results, try Gemini as BYOK. Would love to collaborate with smarter people than me to rig this up with a great opensource candidate, I myself have failed at that.

Also want technical feedback from people messing with prompt-to-code pipelines in 3D generation contexts.

Btw, for fun: Interesting generated examples:
- Boston Dynamics-style robot dog: https://imgur.com/a/CqMYgrF
- A microwave: https://imgur.com/a/hIqIJdr

submitted by /u/mhb-11
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA