What is the InMemoryEmbeddingStore and when should you migrate to a real vector database?

InMemoryEmbeddingStore is LangChain4j's simplest EmbeddingStore implementation: it holds all embeddings in a Java List in heap memory, performs linear scan (brute-force cosine similarity) for similarity search, and has zero external dependencies. It ships in the core module with no additional Maven dependency.

// Zero setup — ready to use in any test or prototype
EmbeddingStore<TextSegment> store = new InMemoryEmbeddingStore<>();

// Serialize to JSON file for lightweight persistence
String json = store.serializeToJson();
Files.writeString(Path.of("embeddings.json"), json);

// Deserialize on next startup
EmbeddingStore<TextSegment> restored =
    InMemoryEmbeddingStore.fromJson(Files.readString(Path.of("embeddings.json")));

It does support basic JSON file persistence via serializeToJson() and fromJson(), so for truly small corpora it can survive restarts — but it is still a single-file, single-node solution.

You should migrate to a real vector database (PgVector, Qdrant, Pinecone, etc.) when any of these conditions are true:

• Scale — More than ~50,000 document chunks. Linear scan becomes visibly slow (~100ms+) at this scale versus ANN index millisecond queries
• Filtering — You need metadata-filtered similarity search (find documents by author AND semantic similarity). InMemoryEmbeddingStore has no filtering support
• Persistence — Multiple pods that need to share the same embeddings. A JSON file cannot serve multiple instances
• Updates — Frequent document additions or deletions. Rebuilding the in-memory store from scratch is expensive for large corpora
• Disaster recovery — If re-embedding your entire corpus on every restart takes more than seconds, the file-based approach is too fragile

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.