What is the InMemoryEmbeddingStore and when should you migrate to a real vector database?

InMemoryEmbeddingStore is LangChain4j's simplest EmbeddingStore implementation: it holds all embeddings in a Java List in heap memory, performs linear scan (brute-force cosine similarity) for similarity search, and has zero external dependencies. It ships in the core module with no additional Maven dependency.

// Zero setup — ready to use in any test or prototype
EmbeddingStore<TextSegment> store = new InMemoryEmbeddingStore<>();

// Serialize to JSON file for lightweight persistence
String json = store.serializeToJson();
Files.writeString(Path.of("embeddings.json"), json);

// Deserialize on next startup
EmbeddingStore<TextSegment> restored =
    InMemoryEmbeddingStore.fromJson(Files.readString(Path.of("embeddings.json")));

It does support basic JSON file persistence via serializeToJson() and fromJson(), so for truly small corpora it can survive restarts — but it is still a single-file, single-node solution.

You should migrate to a real vector database (PgVector, Qdrant, Pinecone, etc.) when any of these conditions are true:

• Scale — More than ~50,000 document chunks. Linear scan becomes visibly slow (~100ms+) at this scale versus ANN index millisecond queries
• Filtering — You need metadata-filtered similarity search (find documents by author AND semantic similarity). InMemoryEmbeddingStore has no filtering support
• Persistence — Multiple pods that need to share the same embeddings. A JSON file cannot serve multiple instances
• Updates — Frequent document additions or deletions. Rebuilding the in-memory store from scratch is expensive for large corpora
• Disaster recovery — If re-embedding your entire corpus on every restart takes more than seconds, the file-based approach is too fragile