How to deal with text only vector search across multimodal embedding space? [D]

My data set is a list of images, each equipped with a a couple sentences of text.

A user would search primarily with text only. My default approach is using BM25, but how would I facilitate searching with a vector DB and a model that embeds vectors in a multimodal combined space?

Here is my dilemma:

Do I embed text part and image part as 2 separate individual vectors or do I combine them into 1 vector?

If a typical search happens with text only, that would immediately deprioritize all image-only embeddings and only good text matches would float up. This is why I am now considering embedding text and images together but would prefer to hear more opinions on this. Thanks.