Prev Next

Database / Weaviate Vector database Interview questions

Explain the data flow of an object being vectorized and indexed in Weaviate?

Inserting an object into a collection with a configured vectorizer involves several steps between the client's insert call and the object becoming fully searchable via its vector index.

sequenceDiagram participant Client participant Weaviate participant Vectorizer as Vectorizer Module (e.g. text2vec-openai) participant Inverted as Inverted Index (BM25) participant HNSW as Vector Index (HNSW) participant Disk as Persistent Storage Client->>Weaviate: Insert object (properties) Weaviate->>Vectorizer: Send text properties for embedding Vectorizer-->>Weaviate: Return vector embedding Weaviate->>Inverted: Index text properties for BM25 Weaviate->>HNSW: Insert vector into graph index Weaviate->>Disk: Persist object, vector, and index updates Weaviate-->>Client: Return object UUID

Once the client submits an object, Weaviate sends its relevant text (or other configured source properties) to the vectorizer module, which returns an embedding. In parallel, Weaviate updates the inverted index used for BM25 keyword search from the object's text properties, and inserts the new vector into the collection's configured vector index (HNSW, Flat, or Dynamic). All of this, the raw object, its vector, and the index structures, gets persisted to disk, and the object becomes searchable via vector, keyword, or hybrid queries once this pipeline completes.

When an object with text properties is inserted into a collection with a configured vectorizer:
Alongside vector indexing, Weaviate also updates:

More Related questions...

What is Weaviate? What is a Collection in Weaviate? What is a vectorizer module in Weaviate? What is hybrid search in Weaviate? What is the HNSW index in Weaviate? What is a Flat index in Weaviate? What is the Dynamic index in Weaviate? What is a cross-reference in Weaviate? What are named vectors in Weaviate? What is Weaviate Cloud? What is quantization/compression in Weaviate, and why is it used? What is Rotational Quantization (RQ) in Weaviate? What is generative search (RAG) in Weaviate? What is multi-tenancy in Weaviate? What is Object TTL in Weaviate? What is the Weaviate Query Agent? What is a UUID's role for objects in Weaviate? What are properties in a Weaviate collection? What is BM25 in the context of Weaviate? What is the alpha parameter in Weaviate hybrid search? What are the main deployment options for Weaviate? What is replication in Weaviate? What is sharding in Weaviate? What are the main use cases for Weaviate? What is the difference between bringing your own vectors and using a vectorizer module? Explain the data flow of an object being vectorized and indexed in Weaviate? Why does Weaviate combine BM25 and vector search instead of using vector search alone? How does Weaviate differ from Milvus? What is the difference between HNSW and the HFresh index in Weaviate? How do you choose between PQ, BQ, SQ, and RQ quantization? When should you use the Dynamic index instead of always using HNSW? How do you troubleshoot poor recall after enabling quantization in Weaviate? What is the difference between rescoring and raw compressed-vector search? How does Weaviate's multi-tenancy isolate tenant data on disk? Explain the internal working of Weaviate's rescoring mechanism for quantized vectors? What is the difference between Weaviate and Pinecone? How do you implement RAG using Weaviate's generative search module? Why use cross-references sparingly, according to Weaviate's own guidance? What is the difference between rankedFusion and relativeScoreFusion in hybrid search? How does the Weaviate Query Agent route natural-language questions across collections? When would you choose multi-vector (ColBERT-style) embeddings over single-vector embeddings? How do you configure replication factor for high availability in Weaviate? What is the difference between Weaviate's REST/GraphQL API and its gRPC API? Explain the lifecycle of a hybrid search query in Weaviate? How do you optimize Weaviate for memory efficiency at large scale? What is the difference between PQ and RQ quantization internally? How does Weaviate decide which shard(s) to query for a given request? Why should you avoid excessive cross-reference traversal in a single query? What is the difference between Weaviate Database (self-hosted) and Weaviate Cloud? How do you troubleshoot a Weaviate collection running out of memory at scale?
Show more question and Answers...


Comments & Discussions