r/LocalLLaMA · · 1 min read

According to DataBricks, pi-coding-agent is ~2x cheaper than CC/Codex, GLM 5.2 on par with Opus 4.8 high

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

According to DataBricks, pi-coding-agent is ~2x cheaper than CC/Codex, GLM 5.2 on par with Opus 4.8 high

https://preview.redd.it/3p60zyf8afch1.png?width=1840&format=png&auto=webp&s=10dcc90945f0db03352239579fca2132d0c90dfa

https://www.databricks.com/blog/benchmarking-coding-agents-databricks-multi-million-line-codebase

tl;dr pi-coding-agent (bash for everything/minimum tools) is up to 2x cheaper and even has higher pass rate according to their own benchmarks across the board. GLM 5.2 is above GPT 5.5 high and xhigh, on par with Opus 4.8 high.

This is yet another "in our use case"-type benchmark but it comes from DBRX who actually trained a sizeable LLM in the past, and I think they know what they do.

I think their analysis makes sense, and GLM 5.2 genuinely do feel on par with Opus 4.6/4.8 for most coding tasks (I only do step-by-step handheld tasks, not full automation with many subagents, though) and slightly below Opus 4.6/4.8 for generic chatting. YMMV.

A caveat I can think of is that CC's prefix also contains built-in tools like Playwright which is often important for visual tasks or emerging (more advanced) tasks like gameplay agent, and that GLM does not natively support image input.

submitted by /u/NandaVegg
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA