According to DataBricks, pi-coding-agent is ~2x cheaper than CC/Codex, GLM 5.2 on par with Opus 4.8 high
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
| https://www.databricks.com/blog/benchmarking-coding-agents-databricks-multi-million-line-codebase tl;dr pi-coding-agent (bash for everything/minimum tools) is up to 2x cheaper and even has higher pass rate according to their own benchmarks across the board. GLM 5.2 is above GPT 5.5 high and xhigh, on par with Opus 4.8 high. This is yet another "in our use case"-type benchmark but it comes from DBRX who actually trained a sizeable LLM in the past, and I think they know what they do. I think their analysis makes sense, and GLM 5.2 genuinely do feel on par with Opus 4.6/4.8 for most coding tasks (I only do step-by-step handheld tasks, not full automation with many subagents, though) and slightly below Opus 4.6/4.8 for generic chatting. YMMV. A caveat I can think of is that CC's prefix also contains built-in tools like Playwright which is often important for visual tasks or emerging (more advanced) tasks like gameplay agent, and that GLM does not natively support image input. [link] [comments] |
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.