r/MachineLearning · · 1 min read

I reduced image-processing token usage by ~95% compared with GPT-4o direct vision, while maintaining roughly the same accuracy.How significant is that?[P]

Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.

I'm testing a new approach for reducing the cost of image-based LLM inference.

I evaluated it on the MOMA Graph benchmark, using 1,315 questions. Compared with using GPT-4o to process the original images directly, I observed approximately:

  • ~95% lower token usage
  • roughly the same accuracy as the GPT-4o direct-image baseline

I'm intentionally not sharing implementation details yet because the method is still under development.

I'm mainly trying to understand how strong the result itself is.

If these numbers hold across larger and more diverse benchmarks, would you consider this a meaningful result in multimodal AI efficiency?

What evidence would you want to see before taking the claim seriously?

For example:

  • more datasets
  • stronger baselines
  • statistical significance
  • latency measurements
  • API cost comparison
  • performance across different models
  • failure-case analysis

I'm especially interested in feedback from people working on multimodal models, VLM efficiency, or inference optimization.

submitted by /u/angelinusbread
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/MachineLearning