How do you do batch processing with LCEL?

The .batch() method on any LCEL chain processes a list of inputs and returns a list of outputs. Under the hood, LangChain runs the inputs concurrently using a thread pool (synchronous) or asyncio tasks (async), subject to an optional concurrency limit.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

chain = ChatPromptTemplate.from_template("Summarise: {text}") | ChatOpenAI() | StrOutputParser()

texts = [
    {"text": "Article 1 text..."},
    {"text": "Article 2 text..."},
    {"text": "Article 3 text..."},
]

# Runs concurrently, returns list in input order
summaries = chain.batch(texts)

# Limit concurrency to avoid rate limits
summaries = chain.batch(texts, config={"max_concurrency": 5})

The async equivalent is .abatch(), which is preferred in async applications:

summaries = await chain.abatch(texts, config={"max_concurrency": 5})

Batch processing is ideal for offline data pipelines: indexing document collections, running evaluations against a test set, or bulk extracting structured data from unstructured text. Results are always returned in the same order as the input list, even if individual tasks complete out of order.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.