I compared local models and different quants / config on a subset of swe-verified bench

And gathered a lot of data. you can see them for yourself

And For the most curious, there are additional details here

In this graph, I regrouped the finetunes under their base models. but you can see the details in the page.

The python code to generate those pages is obviously vibecoded. I find the output kinda pretty and somewhat useful for me. maybe it's useful for someone else.

Heading for a vacation for a few weeks, but if you have any suggestion, I will consider each of them.