AI / LangGraph LangChain Interview questions II
How do you test LangChain applications?
Testing LangChain applications requires strategies for both unit testing individual components without real LLM calls, and end-to-end evaluation of response quality.
Unit testing with fake LLMs — use FakeListLLM or FakeListChatModel to return predetermined responses so tests run fast and deterministically without API calls:
from langchain_community.llms.fake import FakeListLLM from langchain_core.prompts import ChatPromptTemplate from langchain_core.output_parsers import StrOutputParser fake_llm = FakeListLLM(responses=["Paris", "Berlin", "Tokyo"]) chain = ChatPromptTemplate.from_template("{q}") | fake_llm | StrOutputParser() def test_capital_chain(): result = chain.invoke({"q": "Capital of France?"}) assert result == "Paris"
LangSmith evaluations — create a dataset of input/expected-output pairs in LangSmith and run evaluations using built-in evaluators (qa, criteria, labeled_score_string) or custom LLM-as-judge evaluators:
from langsmith.evaluation import evaluate results = evaluate( my_chain.invoke, data="my-golden-dataset", evaluators=["qa"], experiment_prefix="rag-v2-test", )
For integration tests, use pytest with responses or httpx mocks to simulate LLM API responses. Always test that your chain handles empty outputs, malformed JSON from the LLM, and retriever returning zero documents.
More Related questions...
