r/LocalLLaMA · · 2 min read

Stripping terminal noise from agent context via a lazy-loaded local CLI layer. Looking for brutal feedback on this heuristic.

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

When building long-running coding agents, terminal output is one of the fastest ways to poison a context window.

If an agent runs an intensive build, an install command, or a massive search (grep/find), it easily generates hundreds of lines of raw log noise. The agent reads it once to check the status, but you continue to pay for those massive, redundant log tokens on every subsequent turn in the conversation loop.

To fix this, we designed a lightweight CLI layer called Boost to act as a contextual gatekeeper. We wanted a way to lean out context without stripping the agent's ability to debug deep errors.

The Heuristic: Lazy-Loaded Context Pointers

Instead of feeding raw stdout/stderr back to the LLM agent, the CLI layer intercepts the stream:

  1. Noise Truncation: It identifies repetitive patterns, verbose progress bars (like npm or pip install states), and successful compilation blocks.
  2. Semantic Markers: It replaces the raw log text with a lightweight token marker: [Terminal Output Truncated: 450 lines of webpack build logs. Reference ID: log_9a3b. Status: Success]
  3. Lazy Hydration: The raw log is cached 100% locally. If the agent hits a roadblock later and explicitly decides it needs to inspect the compiler output to debug an error, it invokes a sub-command to "hydrate" that specific reference ID (e.g., boost inspect log_9a3b).

In our internal testing, this drastically dropped token consumption per session without degrading task success rates.

The Transparency Part (JFrog & Privacy)

We operate as an internal startup incubated within JFrog. Being backed by a larger organization means we have the dedicated resources to build stable developer tooling, but we are taking this one step at a time.

Let's be 100% transparent about where the tool stands right now:

  • Privacy & Telemetry: It runs 100% locally on your machine and processes truncation entirely in-memory. Your raw terminal outputs, code, and logs never leave your machine. The only network call it makes is a lightweight telemetry ping to track aggregate token savings so we can measure the tool's real-world impact and improve it.
  • The Code: The binary is completely free, but the source code is currently closed while we validate the core mechanics.

You can check it out at boost.jfrog.com or find the releases on our GitHub page: github.com/jfrog/boost

Beyond the tool itself, we want to know if this logic breaks down at scale. How are you keeping your agent context lean without breaking workflows? Are you using custom system prompts, vector embeddings for logs, or just letting the tokens burn?

submitted by /u/Lanky_Hall7250
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA