SigLIP 2 text embedding on CPU with Rust + ONNX

We’re building a robotics data platform with a lot of images, video, and text metadata.

For search, we use SigLIP 2. GPUs handle batched asynchronous image/video embedding and indexing, while this small Rust + ONNX Runtime service handles live text queries on CPU. Both land in the same embedding space, so a text query can search the GPU-built image index.

The split has been practical for us: keep scarce GPUs focused on high-throughput multimodal ingestion, and scale the lighter query path on regular CPU instances.

Curious whether others are using Rust for similarly narrow, latency-sensitive inference services, especially around multimodal or vector-search workloads.

https://github.com/Hebbian-Robotics/siglip-onnx-server