r/MachineLearning · · 1 min read

Do VLMs in production still use fixed-patch ViTs for their vision capabilities? [D]

Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.

The research community has provided (already for some time) seemingly more efficient and effective tokenizations for vision. Do we have any hint on whether non-fixed-patches tokenization is being applied on the big player models?

I imagine not, and I'm trying to think why:

- marginal gains?

- pipelines needing a fixed number of tokens per image upfront for efficiency reasons (or even harder limitations)?

- scaling laws are not well understood for input-adaptive patching therefore big players do not bet on this?

or am I simply totally wrong and under the hood all the big players are doing dynamic tokenization for vision?

submitted by /u/howtorewriteaname
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/MachineLearning