Database / LanceDB Interview questions
How do you implement custom embedding functions in LanceDB?
When a built-in embedding provider in the registry doesn't cover a specific model, LanceDB lets you implement the EmbeddingFunction interface directly and register it, so the custom function gets the same automatic embed-on-insert and embed-on-query behavior as any built-in provider.
from lancedb.embeddings import EmbeddingFunctionRegistry, EmbeddingFunction import numpy as np registry = EmbeddingFunctionRegistry.get_instance() @registry.register("my-custom-model") class MyCustomEmbeddingFunction(EmbeddingFunction): def ndims(self): return 512 def generate_embeddings(self, texts): return [self._embed_one(t) for t in texts] def _embed_one(self, text): # call your own model / API here return np.random.rand(512).tolist()
The interface requires implementing at minimum ndims() (declaring the embedding's fixed dimensionality) and generate_embeddings() (the actual embedding logic, called with a batch of inputs), with the EmbeddingFunctionRegistry handling the details of serializing which function and configuration a table's schema depends on, so that reopening the table later still knows how to regenerate embeddings consistently.
Once registered, a custom embedding function is used in a table schema exactly like a built-in one — via SourceField() and VectorField() — which is what keeps custom and built-in providers interchangeable from the perspective of the rest of the API.
More Related questions...