Cloud / Azure AI search Interview questions

Azure AI Search (formerly Azure Cognitive Search) is a managed search service on Azure. You load your own content into an index, then query it with full-text, vector, hybrid or semantic search over REST or an SDK.

It is used for website and enterprise search, and increasingly as the retrieval layer in RAG (retrieval-augmented generation) apps, where an LLM needs grounded answers from private data.

Three things set it apart from a plain database query: a relevance-tuned query engine, ingestion tools (indexers and skillsets) that pull and enrich data from Azure sources, and built-in scaling and security so you do not run search servers yourself.

An Azure AI Search solution is built from a handful of objects that live on one search service:

• Index - the searchable store of documents and its schema.
• Data source - connection details for the content you want to ingest (Blob Storage, SQL, Cosmos DB and so on).
• Indexer - the crawler that reads a data source and fills the index.
• Skillset - an optional enrichment pipeline (OCR, entity extraction, chunking, embeddings) that runs inside an indexer.
• Synonym maps - query-time term expansion rules.
• Knowledge store - optional storage of enriched output in Azure Storage.

For agentic scenarios you also create knowledge sources and a knowledge base on top of these.

A search index is the persistent, queryable store on your search service. It has a schema (a list of fields with types and attributes) and holds the documents you loaded. Queries never touch your original data source, only the index.

Internally, searchable text fields are kept in an inverted index (term to documents), while vector fields are kept in a dedicated vector index. The same index definition can also carry analyzers, scoring profiles, suggesters, a semantic configuration and vector search settings.

Every index needs exactly one key field of type Edm.String.

{
  "name": "hotels-index",
  "fields": [
    { "name": "hotelId",   "type": "Edm.String", "key": true },
    { "name": "hotelName", "type": "Edm.String", "searchable": true, "sortable": true },
    { "name": "category",  "type": "Edm.String", "filterable": true, "facetable": true },
    { "name": "rating",    "type": "Edm.Double", "filterable": true, "sortable": true }
  ]
}

A search document is a single record in an index, represented as a JSON object whose properties match the index fields. It is the unit that gets indexed, scored and returned in results.

What counts as a document is a design choice. For a product catalog it is usually one product. For a RAG index built from PDFs it is usually one chunk of a PDF, carrying the chunk text, its embedding and metadata such as the source file name.

Each document has a unique key value. Uploading a document whose key already exists replaces or merges into the existing one, depending on the action you choose.

Field attributes are flags on each index field that decide what you can do with it. Turning on only what you need keeps the index smaller and faster.

Each extra attribute adds index size, and attributes on an existing field cannot be switched on later without rebuilding, so decide them up front.

An indexer is a built-in crawler that connects to a supported data source, extracts text and metadata, maps it to index fields and loads the results. It is the pull model of indexing: the service fetches the data, so you write no ingestion code.

Key behaviors:

• Runs on demand or on a schedule (the minimum interval is 5 minutes).
• Tracks state so later runs only pick up new or changed items.
• Applies field mappings, and optionally a skillset for AI enrichment.
• Reports success, warnings and errors in its execution history.

Built-in indexers cover the main Azure data stores:

• Azure Blob Storage and ADLS Gen2 (PDF, Office files, JSON, CSV, HTML, images and more)
• Azure Table Storage
• Azure SQL Database, SQL Managed Instance and SQL Server on Azure VMs
• Azure Cosmos DB
• Microsoft OneLake (Fabric) and SharePoint Online, with some of these still in preview depending on the API version

For anything without a connector, such as an on-premises database or a SaaS API, use the push model: write code (or an Azure Function or Data Factory pipeline) that sends JSON documents to the index.

In the push model your application calls the Index Documents API and sends JSON. In the pull model an indexer fetches the data for you.

Many production systems mix both: indexers for the bulk load and push calls for real-time updates.

A skillset is an ordered set of enrichment steps (skills) that an indexer runs on each document after it has been cracked open and before it is written to the index. This is what Microsoft calls AI enrichment.

Built-in skills include OCR, image analysis, language detection, key phrase and entity extraction, Text Split (chunking), Document Layout and embedding skills. A custom Web API skill lets you call your own code.

Skills read from and write to an enriched document tree, for example /document/content. Outputs reach the index through output field mappings or index projections.

{
  "@odata.type": "#Microsoft.Skills.Text.SplitSkill",
  "context": "/document",
  "textSplitMode": "pages",
  "maximumPageLength": 2000,
  "pageOverlapLength": 200,
  "inputs":  [ { "name": "text", "source": "/document/content" } ],
  "outputs": [ { "name": "textItems", "targetName": "pages" } ]
}

A knowledge store is optional storage in your Azure Storage account for the enriched output of a skillset. It is defined inside the skillset as a set of projections.

• Table projections: shaped rows in Azure Table Storage, good for Power BI.
• Object projections: JSON blobs of the enriched document.
• File projections: extracted images saved as blobs.

It is meant for analytics, data science and debugging enrichment results. It is not queried by the search engine and does not replace the search index.

Full-text search is the lexical (keyword) search mode. The query is run through an analyzer, matched against terms in the inverted index and ranked with BM25. It finds documents that contain the words you typed, not documents that merely mean the same thing.

It supports stemming, phrase queries, wildcards, fuzzy matching (Lucene syntax), field boosting, synonyms and hit highlighting.

POST https://<service>.search.windows.net/indexes/hotels-index/docs/search?api-version=2026-04-01
{
  "search": "beach resort pool",
  "searchFields": "hotelName,description",
  "select": "hotelName,rating",
  "top": 5
}

Vector search finds documents by meaning instead of exact words. An embedding model turns text (or images) into an array of floats, you store it in a vector field, and at query time the service returns the nearest neighbours of the query vector.

A vector field is typed Collection(Edm.Single) with a fixed number of dimensions and a vector search profile that names the algorithm (HNSW or exhaustive KNN) and similarity metric (cosine is common).

The embedding model and dimensions used at query time must match those used at indexing time.

{
  "select": "title,chunk",
  "vectorQueries": [
    { "kind": "text", "text": "how do I reset my password",
      "fields": "contentVector", "k": 5 }
  ]
}

Hybrid search runs a keyword query and one or more vector queries in a single request, in parallel, and merges the ranked lists into one result set using Reciprocal Rank Fusion (RRF).

It covers the weakness of each side. Keywords catch exact tokens such as product codes, names and error numbers. Vectors catch paraphrases and intent. Adding the semantic ranker on top is a common recipe for high-quality RAG retrieval.

{
  "search": "error 0x80070005 install fails",
  "vectorQueries": [
    { "kind": "text", "text": "error 0x80070005 install fails",
      "fields": "contentVector", "k": 50 }
  ],
  "select": "title,chunk",
  "top": 10
}

The semantic ranker is a second-stage reranker that uses Microsoft's language understanding models to reorder the top results of a BM25 or hybrid query by meaning. It does not retrieve anything new, it only reorders what the first stage found.

It adds:

• A @search.rerankerScore from 0 to 4.
• Extractive captions (highlighted passages) and optional answers.

To use it, define a semantic configuration on the index (title, content and keyword fields) and send queryType=semantic with the configuration name. It is billed separately from search units.

You pick a tier when you create the service, and it sets storage, scale limits and price. Switching tiers later generally means creating a new service, so size up front.

Microsoft has also introduced a consumption-based serverless model in preview alongside these dedicated tiers.

Replicas are copies of your index that share query load and give high availability. Partitions split the index into shards, adding storage and faster indexing.

A request that fans out to several partitions is merged by the service, and replicas of the same partition answer in a load-balanced way.

A search unit is the billing and capacity unit: one replica times one partition. The formula is SU = replicas x partitions, so 2 replicas and 3 partitions is 6 SUs.

You pay an hourly rate per SU for the tier you chose, for as long as the service exists. There is no pause button, so an idle service still bills until you delete it.

Some costs sit outside SUs: built-in AI skills such as OCR are billed through an Azure AI services resource, semantic ranker usage is metered on its own, and models used for agentic query planning bill on your model deployment.

There are two inbound options, plus managed identity for outbound calls:

• API keys - two admin keys (full control) and up to 50 query keys (read-only on documents).
• Microsoft Entra ID with Azure RBAC - roles such as Search Service Contributor, Search Index Data Contributor and Search Index Data Reader.
• Managed identity - lets indexers and skills reach Storage, SQL or Azure OpenAI without connection-string secrets.

RBAC is the better choice for production. You can disable key-based access entirely once your apps use Entra tokens.

An analyzer turns raw text into the tokens stored in the inverted index, and does the same to query text so both sides line up. It runs three stages in order: character filters, a tokenizer, then token filters (lowercasing, stop words, stemming).

The default is the standard Lucene analyzer. You can switch to a language analyzer such as en.microsoft or en.lucene for stemming, or define a custom analyzer.

Set it per field with analyzer, or split it with indexAnalyzer and searchAnalyzer. Changing it on an existing field needs a rebuild. The Analyze Text API shows the tokens an analyzer produces.

A scoring profile is part of the index definition that tweaks how text-query relevance scores are calculated. It lets you push certain matches up without changing the data.

• Field weights - a match in title counts more than one in body.
• Functions - freshness (newer first), magnitude (higher numbers first), distance (nearer first), tag (matching tags first).

Apply it per request with scoringProfile, or make it the index default. Scoring profiles adjust BM25-style scores for text queries; they do not change vector similarity.

{
  "name": "boost-recent-titles",
  "text": { "weights": { "title": 3, "body": 1 } },
  "functions": [
    { "type": "freshness", "fieldName": "publishedOn", "boost": 2,
      "parameters": { "boostingDuration": "P30D" } }
  ]
}

A synonym map is a service-level resource that lists terms that should be treated as equivalent. You attach it to a searchable field, and the engine expands the query at search time. The index itself is not changed.

It uses the Solr format. Equivalent terms are comma-separated; an arrow maps several terms to one:

USA, United States, United States of America
Washington, WA => WA

Because expansion happens at query time, you can edit a map and see the effect without rebuilding the index. Synonyms apply to text queries, not to vector queries.

Both power type-ahead search boxes, working on partial input as the user types.

• Suggest API - returns matching suggestions, optionally with the documents that matched, for a prefix like "lux".
• Autocomplete API - returns completed terms or short phrases (for example "luxury" or "luxury hotel") and no documents.

Both depend on a suggester declared in the index definition, with a list of sourceFields to draw suggestions from. Autocomplete has modes (oneTerm, twoTerms, oneTermWithContext) that control how much is completed.

Facets give the category counts you see in the sidebar of a shopping site. Mark the field facetable, then request it with the facet parameter. The response includes @search.facets with a count per value.

POST /indexes/hotels-index/docs/search?api-version=2026-04-01
{
  "search": "*",
  "facets": [ "category,count:5", "rating,interval:1" ],
  "filter": "rating ge 3"
}

When the user clicks a value, send the same query again with a $filter such as category eq 'Resort'. With many partitions, counts on high-cardinality fields can be approximate, so raise count if accuracy matters.

Agentic retrieval is a multi-query pipeline designed to feed LLMs and agents. An LLM first plans the retrieval: it uses the conversation history to break a complex question into focused subqueries. Those run in parallel across your sources, results are semantically reranked, and everything is merged into one grounding response with references.

It beats a single search call for questions such as "compare the refund policy with the warranty terms I asked about earlier", where one query would miss half of the intent.

The trade-offs are extra latency from the planning step and LLM token charges on your own model deployment. Check the current docs, since the 2026-04-01 REST API covers generally available knowledge source types while newer ones remain in preview.

These are the two building blocks of agentic retrieval.

A knowledge source points at content. An indexed source is ingested into a search index for you (for example from a search index, Blob Storage, OneLake, Azure SQL or uploaded files). A remote source is queried live with no indexing (for example the web, an MCP server or a Microsoft 365 or Fabric source).

A knowledge base is the top-level object that groups one or more knowledge sources with the chat model and retrieval instructions. Your app (or a Foundry agent) calls the knowledge base's retrieve action, and the service plans, queries and merges across all its sources.

Many source types are still preview, so verify availability before committing a design.

Both parse the search text, but simple (the default) is forgiving and meant for user-typed input, while full Lucene (queryType=full) is stricter and exposes advanced operators for developers.

Wildcard and regex terms skip analysis, so stemming and synonyms are not applied to them.

BM25 scores a document for a query by combining three ideas:

1. Term frequency with saturation - more occurrences help, but with diminishing returns so repeating a word 50 times does not win.
2. Inverse document frequency - rare terms count more than common ones.
3. Length normalization - a match in a short field counts more than the same match buried in a long one.

Two parameters tune it at the index level: k1 (saturation, default 1.2) and b (length normalization, default 0.75). A higher b penalizes long fields more.

IDF is computed per shard, so tiny indexes spread over several partitions can rank slightly differently between runs. Setting scoringStatistics=global on the query makes the stats consistent.

RRF ignores raw scores and uses only ranks. Each retriever (the keyword query and every vector query) produces its own ranked list, and a document's fused score is the sum of 1 / (k + rank) across the lists it appears in. k is a smoothing constant, commonly shown as 60.

Ranks are used because BM25 scores and cosine similarities live on different scales and cannot be added meaningfully.

Document A wins because both retrievers liked it. Vector queries can also carry a weight to tilt the fusion, and the final @search.score is the RRF value, which is why it looks tiny.

The semantic ranker works as a second stage after normal retrieval:

1. The first stage (BM25, vector or hybrid) returns candidates; the top 50 go to the ranker.
2. For each candidate, the text of the fields in your semantic configuration is assembled (title first, then content fields in priority order) and truncated to the model's input limit.
3. A language model scores how well each document answers the query and reorders them.
4. It returns @search.rerankerScore, plus optional captions and answers.

"semantic": { "configurations": [ {
  "name": "default-semantic",
  "prioritizedFields": {
    "titleField": { "fieldName": "title" },
    "prioritizedContentFields": [ { "fieldName": "chunk" } ]
  } } ] }

// query: queryType=semantic, semanticConfiguration=default-semantic,
//        captions=extractive, answers=extractive|count-3

Since text beyond the input limit is cut, put the most informative field first.

Both find nearest neighbours but trade speed against exactness.

efSearch controls how many candidates are examined at query time: higher means better recall and more latency. Use exhaustive KNN for small sets, for heavy filtering, or to build a ground truth for measuring HNSW recall. A query can also set exhaustive: true on an HNSW field for that request only.

Integrated vectorization moves chunking and embedding inside Azure AI Search, so you do not run your own pipeline code.

At indexing time, an indexer reads the files, a Text Split skill chunks them, an embedding skill (for example Azure OpenAI Embedding) vectorizes each chunk, and index projections write one search document per chunk linked to its parent.

At query time, a vectorizer defined in the index turns the user's text into a vector, so clients send plain text with kind: text.

Why use it: fewer moving parts, embeddings stay in sync through the indexer's change detection, and the query side needs no separate embedding call.

flowchart LR
  A["Data source"] --> B["Indexer cracks files"]
  B --> C["Text Split skill"]
  C --> D["Embedding skill"]
  D --> E["Index projections"]
  E --> F[(Chunk index)]
  Q["User text query"] --> V[Vectorizer]
  V --> F

It depends on the data source. Indexers keep a high-water mark or change token between runs and only fetch what moved.

With a high-water-mark policy alone, rows physically deleted from the source are invisible to the indexer, which is why a soft-delete column is recommended. You can force a full pass by resetting the indexer.

Start by finding the bottleneck: source reads, enrichment skills or the service itself.

1. Batch and parallelize pushes - send batches (often 100 to 1,000 docs) from several threads, and tune batch size by testing.
2. Back off on 503 and 429 - retry with exponential backoff instead of hammering the service.
3. Add partitions - they raise indexing throughput and storage together.
4. Run several indexers in parallel - split the source (for example by blob prefix or SQL key range) across multiple data sources and indexers.
5. Trim the index - drop filterable, sortable or facetable from fields that do not need them.
6. Enable the enrichment cache - so a skillset change only reruns the affected skills, not the whole corpus.
7. Tune custom Web API skills - raise batch size and degree of parallelism if your endpoint can take it.

Vector indexes are memory-heavy, so most savings come from storing less per vector.

Compression costs some recall. Offset it with oversampling and rescoring against the original vectors, and measure recall on your own queries before and after.

Match the move to the symptom.

Remember that SUs multiply, so going from 3x1 to 3x2 doubles the bill. Adding partitions reshuffles shards in the background, which takes time, so plan it ahead of a big load rather than during one.

Most gains come from asking the service to do less.

• Use $select to return only the fields you display, and keep top small.
• Limit searchFields to the fields that matter instead of searching every searchable field.
• Prefer $filter on filterable fields for exact constraints; filters are cheap compared to scoring text.
• Avoid leading wildcards and heavy regex over large indexes.
• Keep vector k and efSearch only as high as your recall target needs.
• Reuse one SDK client (connection reuse) and run the app in the same region as the service.
• Add replicas for throughput, and watch search latency and throttled-query metrics in Azure Monitor.

The classic approach is a security filter. Store the allowed principals on every document, then filter on the caller's identity at query time.

1. Add a field such as group_ids typed Collection(Edm.String) and mark it filterable.
2. At indexing, fill it with the group or user IDs allowed to see the document.
3. At query time, your backend resolves the caller's groups (for example from Entra ID) and appends the filter.

$filter=group_ids/any(g: search.in(g, 'sales-team,emea-staff,all-users'))

Never let the browser build or override this filter. Microsoft is also adding preview support for enforcing source ACLs and RBAC scopes from stores such as ADLS Gen2, so check current docs before building it by hand.

Think in two directions: who can reach the service, and what the service can reach.

Typical locked-down setup: private endpoint for the app, shared private links for each data source, and a managed identity with the right Storage and OpenAI roles.

The two attributes build different structures from the same text.

For a category value of Resort Hotel, a search for resort matches. The filter category eq 'Resort Hotel' matches, but category eq 'resort hotel' does not. Fields used both ways cost more space, so enable both only when you really need them.

Each request returns at most $top = 1,000 results, and $skip tops out at 100,000. Skipping is costly because every shard must produce and sort all the earlier results just to throw them away, so the service caps it.

The workaround is keyset pagination: sort on a unique sortable field and ask for items after the last one you saw.

// page 1
$orderby=id asc&$top=1000
// next page: last id of previous page was 'a1042'
$filter=id gt 'a1042'&$orderby=id asc&$top=1000

For one-off exports, split the data by a filterable range (date, category) and loop. Users rarely read past a few pages anyway, so refining the query is usually the better UX fix.

Azure AI Search lets you add new fields to a live index, but not alter existing ones in ways that affect the physical index.

• Allowed in place: adding a field, scoring profiles, CORS settings and synonym map assignments. Existing documents simply have null in the new field until you reload them.
• Needs a rebuild: changing a field's type, its analyzer, or turning on attributes like filterable or sortable for a field that already exists.

A rebuild means dropping and recreating the index, then reloading all data. To avoid downtime, build a new index version beside the old one and switch with an alias, as described in the next topic.

Use an index alias. The alias is a stable name that points at one real index, and clients query the alias instead of the index name.

1. Create products-v1 and an alias products pointing to it.
2. When the schema must change, create products-v2 next to it.
3. Load products-v2 with an indexer or push job while users keep hitting v1.
4. Validate counts and test queries on v2.
5. Update the alias to target products-v2, which takes effect immediately.
6. Delete products-v1 after a rollback window.

flowchart LR
  App["Client app"] --> Alias["alias: products"]
  Alias -. before swap .-> V1[(products-v1)]
  Alias == after swap ==> V2[(products-v2)]
  Job["Indexer / push job"] --> V2

A single call to a knowledge base's retrieve action triggers the whole pipeline.

1. The app sends the conversation (recent messages) to the knowledge base.
2. The configured LLM plans the query: it rewrites and splits the request into subqueries, guided by any retrieval instructions.
3. Subqueries run in parallel against the knowledge sources; indexed sources use hybrid (text plus vector) search.
4. The semantic ranker reranks each subquery's results.
5. Results are merged into one grounding payload, with references and an activity log showing what was planned and run.
6. Your app (or agent) passes the grounding data to an LLM to write the answer. Depending on settings, the knowledge base can also synthesize an answer itself.

sequenceDiagram
  participant App
  participant KB as Knowledge base
  participant LLM as Planning LLM
  participant KS as Knowledge sources
  App->>KB: retrieve(messages)
  KB->>LLM: plan subqueries
  LLM-->>KB: subqueries
  par each subquery
    KB->>KS: hybrid / remote query
    KS-->>KB: ranked results
  end
  KB->>KB: semantic rerank and merge
  KB-->>App: grounding data, references, activity log

One run follows a fixed order:

1. Connect to the data source and list items that are new or changed since the last high-water mark.
2. Document cracking - open each file, pull out text, metadata and images.
3. Field mappings copy source fields into the enriched document (for example /document/content).
4. Skillset - skills run in dependency order and add nodes to the enriched document tree. Results can be cached.
5. Output field mappings (or index projections) move enriched nodes into index fields.
6. Documents are written to the index; optional projections go to the knowledge store.
7. The indexer saves its checkpoint and logs successes, warnings and errors.

flowchart TD
  A["Data source"] --> B["Document cracking"]
  B --> C["Field mappings"]
  C --> D["Skillset enrichment"]
  D --> E["Output field mappings / projections"]
  E --> F[(Search index)]
  D --> G[(Enrichment cache)]
  D --> H[(Knowledge store)]

Azure AI Search is the retrieval half of RAG. The LLM never searches by itself; your app retrieves relevant chunks first and hands them to the model as context.

1. Chunk and embed documents into an index (integrated vectorization or your own pipeline).
2. At question time, run a hybrid query with the semantic ranker and take the top few chunks.
3. Build a prompt: instructions, the retrieved chunks with their sources, then the question.
4. Call the Azure OpenAI chat model and return the answer with citations.

from azure.search.documents.models import VectorizableTextQuery

results = client.search(
    search_text=question,
    vector_queries=[VectorizableTextQuery(
        text=question, k_nearest_neighbors=50, fields="contentVector")],
    query_type="semantic",
    semantic_configuration_name="default-semantic",
    select=["title", "chunk"],
    top=5,
)
context = "\n\n".join(f"[{r['title']}] {r['chunk']}" for r in results)

Azure OpenAI On Your Data and Foundry agents use the same service underneath, so you can start with those or code the loop yourself for control.

Choose Azure AI Search when search itself is the product feature, not just a lookup on a column. Database full-text search is fine when the need is small and tightly tied to transactional data.

A simple keyword lookup over one SQL table with modest data is a good case for staying in the database. A multi-source, relevance-sensitive or RAG scenario is where Azure AI Search pays off.

Work from the outside in.

1. Read the execution history (portal or Get Indexer Status). Compare itemsProcessed with itemsFailed and read the first error or warning.
2. Check connectivity - connection string or managed identity roles, firewall, shared private link.
3. Check the source scope - container and folder path, SQL query, and indexedFileNameExtensions or excludedFileNameExtensions filters on blob indexers.
4. Check change detection - the high-water mark may say nothing is new. Reset the indexer and rerun to force a full pass.
5. Check deletion policy - a soft-delete column that is true for every row will skip everything.
6. Check mappings and parsing - wrong parsingMode, a missing key mapping or a type mismatch.
7. Use a debug session for skillset problems to inspect the enriched tree.

Change one thing at a time and keep a set of test queries with the results you expect.

1. Check tokenization with the Analyze Text API. A wrong analyzer is a common cause of misses.
2. Check which fields are searched - use searchFields and field weights so titles outrank body text.
3. Check searchMode - any (default) is broad, all is strict.
4. Add synonyms for vocabulary gaps.
5. Try hybrid and semantic ranking if keyword matches miss intent.
6. Review chunking for RAG: chunks that are too large or too small hurt retrieval.
7. Rule out noise - on small indexes with several partitions, use scoringStatistics=global, and a sessionId to keep results steady across replicas.

It means the service cannot take the request right now, usually because indexing or query load exceeds what the current replicas and partitions can handle. It is a signal to slow down, not a permanent failure.

What to do:

1. Retry with exponential backoff and jitter. The Azure SDKs include a retry policy; custom REST clients need their own.
2. Check metrics such as search latency and the percentage of throttled queries.
3. Fix the cause - add replicas for query spikes, add partitions or smaller batches for indexing, and remove expensive queries.

Related: an index batch can return HTTP 207, meaning some documents succeeded and others failed. Read the per-document status in the response and retry only the failed ones.

Chunking decides what the retriever can return, so it deserves testing, not guessing.

• Split with the Text Split skill using maximumPageLength and pageOverlapLength. Newer versions can count tokens rather than characters.
• Respect structure. The Document Layout skill outputs headings and sections, so you can chunk by section instead of cutting mid-sentence.
• Overlap a little (often 10 to 20%) so facts at a boundary are not lost.
• Carry metadata - title, source and page number go into every chunk for citations and filters.
• Use index projections to store one document per chunk linked to a parent.

A few hundred tokens with light overlap is a common starting point. Then compare variants on real questions and keep what retrieves best.