MeshFlow: An open-source orchestrator for governed, cost-optimized multi-agent workflows [D]

Hey ML community, We’ve just open-sourced **MeshFlow** , a code-first, framework-agnostic runtime designed for governing and optimizing multi-agent systems in production. Most agent frameworks focus on rapid prototyping, but ML and platform engineering teams usually run into hard bottlenecks around LLM cost scaling, evaluation alignment, and execution safety. MeshFlow tackles these from a runtime/infrastructure perspective. Here are the key ML and system features: * **Task-Based Model Routing** : Before an agent executes a node, MeshFlow runs an evaluation on task complexity, routing the execution to one of four model tiers (`nano`, `small`, `medium`, `large`). This cuts overall API costs by 50-60% by utilizing smaller local models (e.g. LLaMA-3-8B) for standard formatting or extraction and reservation of frontier models (e.g. Claude Opus) for high-complexity reasoning. * **Context Compactor & Summary Pruning Middleware** : Implements sliding window summarization and context deduplication across parallel agent teams to limit prompt length growth. * **System Prompt Caching** : Native injection of Anthropic `cache_control` tags when system prompts exceed 1024 tokens. * **Cost Regression Evaluation Gate** : Integrates with CI pipelines to evaluate agent changes against a golden scenario baseline, throwing failures if code updates introduce token cost regressions. * **Resilient State Persistence** : Multi-backend state serialization (Redis, PostgreSQL, S3) that preserves checkpoint frames and allows resuming paused workflows. Here is the basic API contract: ```python from meshflow import Workflow, Agent, CostCap wf = Workflow(cost_cap=CostCap(usd=5.00)) wf.add(Agent('researcher'), Agent('critic'), Agent('writer')) result = wf.run('Compile comparative literature review of LLM reasoning pathways') print(result) ``` We'd love to discuss: 1. How do you handle token budget enforcement and model routing in your agent loops? 2. What evaluation pipelines do you use to detect cost or performance regression in production? GitHub: https://github.com/Anteneh-T-Tessema/meshflow