I wanted a quick calories counter for myself, using LLMs to evaluate the calories from pictures of meals + descriptions.

I needed to pick a model so I made a quick benchmark.

The setup was:

- Nutrition5k photos for photo + calories: https://github.com/google-research-datasets/Nutrition5k

- A tool with access to calories information from USDA FoodData Central + MEXT

- I evaluated models based on how many of the meals they managed to have under 20% of error

- All on the same randomly picked 25 meals.

Models too big for my machine were run through OpenCode Go/OpenRouter. I've also included Spark 1.3 since it'll supposedly be open weights.

Results

I know it's not the most scientific benchmark, but it's interesting to see that the order is not really linked to model size.

The most interesting for me is how Muse Glimmer 30b trounces Qwen 3.8 27b here. I think it highlights how "the best" model on consumer hardware (~32Gb VRAM) really depends on the task.