How do you implement multimodal search across text and images in LanceDB?

Multimodal search across text and images relies on using a shared embedding model — like CLIP — that maps both modalities into the same vector space, so a text query's embedding can be compared directly against stored image embeddings (or vice versa) using ordinary vector similarity search.

from lancedb.embeddings import get_registry
from lancedb.pydantic import LanceModel, Vector

registry = get_registry()
clip = registry.get("open-clip").create()

class ImageDoc(LanceModel):
    image_uri: str = clip.SourceField()
    vector: Vector(clip.ndims()) = clip.VectorField()

table = db.create_table("images", schema=ImageDoc)
table.add([{"image_uri": "photo1.jpg"}, {"image_uri": "photo2.jpg"}])

results = table.search("a dog playing in the park").limit(5).to_list()

Because CLIP-style models are trained specifically so that a text description and a semantically matching image end up close together in the same embedding space, a plain text query string like "a dog playing in the park" can be embedded and compared directly against stored image vectors, returning visually relevant images even though the query itself was never an image.

This same pattern extends to storing both text and image embeddings in the same or related tables — product descriptions alongside product photos, or document text alongside diagram images — letting one query surface relevant results regardless of which modality the original content was actually captured in.