Cloud / Azure AI Foundry Interview questions

Microsoft Foundry, previously called Azure AI Foundry, is Azure's managed platform for building, deploying and operating AI applications and agents. Model access, an agent runtime, tools, evaluation and governance sit under one Azure resource provider, so RBAC, networking and policy behave the same way across all of it.

In day-to-day work you use it to:

• pick and deploy models from a catalog
• build agents with built-in tools and knowledge sources
• test, evaluate and trace those agents
• apply guardrails and monitor them in production

You reach it through the portal at ai.azure.com, the azure-ai-projects SDK, or REST.

They are successive names for the same product line, not three different services. The portal started as Azure AI Studio, was renamed Azure AI Foundry at Ignite 2024, and became Microsoft Foundry at Ignite in November 2025.

Some URL slugs, package names and screenshots still carry the older brand, so expect a mix when reading documentation.

A Foundry resource is the top-level Azure resource that holds governance settings: networking, security, model deployments and connections. Under the hood it is a Microsoft.CognitiveServices/accounts resource of kind AIServices.

Projects are created beneath it. Services such as Storage, Key Vault and Azure AI Search stay as separate Azure resources that the Foundry resource references through connections, each with its own governance boundary.

Resource-level metrics, such as token consumption, latency and error rates, roll up across every project. An existing Azure OpenAI resource can be upgraded into a Foundry resource while keeping its endpoint and keys.

A Foundry project is the unit of development and isolation inside a Foundry resource. Agents, files, evaluations and connections live in a project, and data is isolated between projects, which makes it handy for separating teams or environments.

Each project has its own endpoint:

https://<resource-name>.services.ai.azure.com/api/projects/<project-name>

You can grant RBAC at the project level and, for stricter isolation or compliance, bring your own Storage, Cosmos DB and Azure AI Search instead of using Microsoft-managed ones.

A common pattern is one project per application or environment (dev, test, prod), so evaluations, files and agent versions from one never leak into another. Creating a project is lightweight compared with creating a new resource, which is why teams usually make several.

The model catalog is the searchable library inside Foundry where you discover, compare and deploy models. Each model has a model card showing capabilities, supported deployment types and regional availability.

It covers models sold directly by Azure (Azure OpenAI models, DeepSeek, xAI and others), partner models such as Mistral, and open or community models, including ones from Hugging Face.

Before deploying, check the card for which deployment types a model supports, because not every model offers every option, such as provisioned or batch.

These are models that Microsoft hosts and bills for under its own product terms, so they get Azure's enterprise commitments and support. They include Azure OpenAI models and others such as DeepSeek and xAI models.

When procurement or compliance matters, this distinction is often what decides the model choice.

Foundry Agent Service is a fully managed platform for building, deploying and scaling AI agents. It provides the runtime that hosts agents, the tools they can call, and connections to models from the Foundry catalog, so you don't operate that infrastructure yourself.

It covers the whole loop: create an agent, test it in the playground, deploy and publish it, then monitor and evaluate it. Private networking is available for prompt and workflow agents.

The agent runtime hosts both prompt agents and hosted agents, and built-in tools include web search, file search, code interpreter, memory and MCP servers.

There are three: prompt agents, workflow agents and hosted agents. They differ in how much you define through configuration versus code.

At the time of writing, workflow and hosted agents were in preview, so confirm their status in the current docs before committing to them.

A prompt agent is a declaratively defined single agent: you give it a model, instructions and tools, and Foundry runs it for you. There is no code to host. You can edit, version, test, evaluate and publish it from the Agents playground or through the SDK and REST.

Each save creates a new version, so you can compare or roll back. Here is a minimal sketch with the Python SDK:

from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition
from azure.identity import DefaultAzureCredential

project = AIProjectClient(endpoint=ENDPOINT, credential=DefaultAzureCredential())
agent = project.agents.create_version(
    agent_name="billing-helper",
    definition=PromptAgentDefinition(
        model="gpt-4o",
        instructions="Answer billing questions in two sentences.",
    ),
)

The SDK surface changes between releases, so check the version you installed.

Agent Service ships several tools you attach by configuration:

• Web search for current public information (preview at the time of writing)
• File search to retrieve from uploaded documents
• Code interpreter to run Python for calculations and data work
• Memory to keep information across sessions (preview)
• MCP servers to reach external tools over Model Context Protocol
• Azure AI Search and Foundry IQ knowledge bases for grounding

For anything else, add custom tools through function calling or an OpenAPI spec.

Attach only what the agent really needs. Every extra tool adds schema text to each model call, which costs tokens and gives the model more ways to choose badly. Preview tools can change, so check their status before relying on them in production.

A deployment is a named, callable instance of a model inside a Foundry resource. It carries its own deployment type, capacity (such as tokens-per-minute quota), model version and assigned guardrail.

Your code calls the deployment name, not the underlying model name, so you can swap the model behind it without changing the application.

openai = project.get_openai_client()
resp = openai.responses.create(
    model="gpt4o-prod",   # deployment name
    input="Summarize this ticket in one line.",
)
print(resp.output_text)

Because capacity and guardrails belong to the deployment, two deployments of the same model can differ in cost and safety behavior. A common habit is one deployment per environment, such as gpt4o-dev and gpt4o-prod, with different quotas.

The deployment type decides where requests are processed, how you pay and how throughput is guaranteed.

Microsoft suggests Global Standard as the starting point unless residency or latency rules say otherwise.

Foundry IQ is a managed knowledge layer for agents, built on top of Azure AI Search. Instead of wiring your own RAG pipeline for every agent, you create a knowledge base once and let agents query it.

A knowledge base is made of knowledge sources (connections to places like Blob Storage, SharePoint, OneLake, search indexes or the web) plus settings that control retrieval. When queried, it uses agentic retrieval to plan searches, respect user permissions and return grounded answers with citations.

Agents can use it through Agent Service, Microsoft Agent Framework, or an MCP endpoint.

A guardrail is a named collection of controls that screens prompts and responses for risk. It is the current name for what used to be called a content filter.

Typical controls cover:

• harm categories (hate, sexual, violence, self-harm) graded safe, low, medium or high
• Prompt Shields for direct and indirect prompt attacks
• protected material detection
• groundedness detection
• custom blocklists

Each control can block or just annotate, and a guardrail is assigned to model deployments and, for some controls, to agents.

Guardrails reduce risk but don't remove it, so pair them with clear instructions, least-privilege permissions and regular evaluation.

The playground is the interactive space in the Foundry portal for trying things before you write code. After you deploy a model you land in the model playground to chat with it, change parameters like temperature, and view the equivalent code.

The Agents playground does the same for agents: edit instructions, attach tools or knowledge, run test conversations, inspect tool calls, and save a new version.

It's for fast iteration and demos, not load testing, so use evaluations and tracing for anything you plan to ship.

Install the SDK, take the project endpoint from the portal, and sign in with Entra ID rather than a key.

1. pip install azure-ai-projects azure-identity
2. Copy the project endpoint from the project overview page.
3. Make sure your identity has the Foundry User role (formerly Azure AI User) on the resource.
4. Create an AIProjectClient with DefaultAzureCredential.

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

project = AIProjectClient(
    endpoint="https://<resource>.services.ai.azure.com/api/projects/<project>",
    credential=DefaultAzureCredential(),
)

If the call fails, check the endpoint and role first. Most connection problems are a wrong project URL, a missing Foundry User assignment, or a sign-in under the wrong tenant. In CI or on Azure, use a managed identity or service principal rather than your personal login.

azure-ai-projects is the main Python SDK for Foundry, built around AIProjectClient. One client gives you access to agents, connections, evaluations and deployments in a project, and can hand you an OpenAI-compatible client for inference.

The 2.x line is the one recommended for new work. It follows the Responses-style model of conversations, items and responses, which differs from the older thread-and-run style used by earlier agent APIs.

That change matters when you read older tutorials: code written for 1.x or for the classic Assistants-style agents often needs porting. Equivalent SDKs exist for .NET, JavaScript and Java.

Foundry uses Azure RBAC with roles that were recently renamed: Foundry User, Foundry Owner, Foundry Account Owner and Foundry Project Manager. They used to be Azure AI User, Azure AI Owner, Azure AI Account Owner and Azure AI Project Manager.

Give app identities the lowest role that works, normally Foundry User, and confirm exact permissions in the current role definitions.

The Foundry Control Plane is the operations layer for AI fleets. It gives you observability, guardrails, policy controls and security in one place, and it works for agents built outside Foundry as well as inside it.

Its main pieces:

• continuous evaluation with agent-specific evaluators such as task adherence and tool-call accuracy
• end-to-end tracing based on OpenTelemetry
• runtime guardrails, including protection from direct and indirect prompt attacks
• a view of agents, models and tools in the Operate area of the portal

Pricing is tied to usage of observability, guardrails and the Microsoft security services involved.

Foundry Local runs AI models on the user's own device instead of in Azure. It exposes a local, OpenAI-compatible endpoint, so code you wrote against a cloud deployment can often be pointed at it with little change.

People use it for offline scenarios, privacy-sensitive data that must not leave the machine, and cheap prototyping without spending tokens. Quality is bounded by the model size your hardware can handle.

It can also be part of a hybrid design: a local agent can still query a cloud Foundry IQ knowledge base when needed.

Hub-based projects are the older architecture, built on an Azure AI hub that owned shared resources and connections. Foundry projects are the newer model, created directly under a Foundry resource, and are the recommended choice today.

The practical rule: new agent features land on Foundry projects only, so plan a migration if you are still on hubs and need them.

If you still depend on classic ML features such as training jobs, check what the newer model supports before moving. For agent-focused work there is little reason to stay on hubs.

Both are pay-per-token. The difference is where your prompts and completions may be processed.

Note that for both, an outage in the primary region affects the traffic initially routed there, so don't treat either as a multi-region failover plan on its own.

Many teams start with Global Standard in development and move production to Data Zone Standard once legal confirms the residency rules. The code change is just the deployment name.

Standard is pay-as-you-go: you pay per token and share capacity, limited by a quota. Provisioned reserves processing capacity measured in PTUs (provisioned throughput units), which you pay for whether or not you use it.

Quota is the other difference worth remembering. Standard quota is set per deployment and region, while PTUs must be sized against your expected tokens-per-minute. Sizing too low still gives 429s and too high wastes budget, so load-test with realistic prompts first.

A prompt agent is configuration: instructions, a model and tools. Foundry runs it. A hosted agent is your code, packaged as a container and run by Agent Service, built with Agent Framework, LangGraph, the OpenAI Agents SDK or your own code.

Hosted agents can still be invoked, evaluated, monitored and published. Start with a prompt agent and move to hosted only when you need custom orchestration or a specific framework.

A quick test: if you can describe the agent as a paragraph of instructions plus a few tools, it is a prompt agent. If you find yourself writing loops, custom state handling or pulling in another framework, it belongs in a hosted agent.

An Azure OpenAI resource gives access to OpenAI models only. A Foundry resource covers those models plus other catalog models, Agent Service, evaluation and the Foundry tools, under one governance boundary.

You do not have to start over to move: an existing Azure OpenAI resource can be upgraded to a Foundry resource while keeping its endpoint, API keys and state, and Azure Policy scoped at that level carries forward.

If your workload only needs completions from OpenAI models and no agents or evaluation, a standalone Azure OpenAI resource may still be enough. Choose Foundry when you want one resource for models, agents and governance.

A run starts when your app sends input to the agent, usually inside a conversation that holds the history. The agent sends the instructions, history and tool definitions to the model deployment. If the model asks for a tool, Agent Service runs it (or hands a function call back to your code), feeds the result to the model, and repeats until the model produces a final answer.

sequenceDiagram
    participant App
    participant Agent as Agent Service
    participant Model as Model deployment
    participant Tool
    App->>Agent: Input + conversation id
    Agent->>Model: Instructions, history, tool schemas
    Model-->>Agent: Tool call request
    Agent->>Tool: Execute tool
    Tool-->>Agent: Tool output
    Agent->>Model: Output added to context
    Model-->>Agent: Final answer
    Agent-->>App: Response

1. Guardrails screen the input and the output.
2. Function tools pause the run: your code runs the function and returns its output.
3. Traces record each model and tool step for later review.

Two points are easy to miss. The conversation object keeps state on the service side, so you don't resend history yourself. And one run can loop through several tool calls before finishing, so set sensible limits and timeouts on your tools.

Plain RAG runs one query and hopes. Agentic retrieval uses an LLM to plan the search first. It looks at the question and chat history, breaks it into focused subqueries, picks the right knowledge sources, runs the searches in parallel, reranks and merges results, and can iterate if the evidence looks thin.

flowchart LR
    A["User question + chat history"] --> B["LLM query planning"]
    B --> C1["Subquery 1"]
    B --> C2["Subquery 2"]
    C1 --> D["Knowledge sources"]
    C2 --> D
    D --> E["Semantic rerank and merge"]
    E --> F{Enough evidence?}
    F -- No --> B
    F -- Yes --> G["Grounded answer with citations"]

A configurable retrieval reasoning effort trades latency and cost for depth. Results respect the caller's permissions where the source supports it.

Compared with a single vector query, this helps on multi-part questions, such as comparing two policies held in different systems, because each part gets its own subquery. The cost is extra LLM calls for planning and merging, so use a lower reasoning effort for simple lookups.

An index is one searchable store of content. A knowledge base sits above one or more knowledge sources, including indexes, and decides how to query them.

An existing index can be registered as a knowledge source, so adopting a knowledge base doesn't mean throwing your index away.

Whichever you choose, security trimming and the freshness of the underlying data still decide answer quality more than the retrieval wrapper does.

A guardrail checks twice: on the way in and on the way out. The prompt is screened against the configured controls before the model sees it. If it passes, the model generates, and the completion is screened again.

flowchart TD
    A[Prompt] --> B["Input checks: harm categories, Prompt Shields, blocklist"]
    B -->|Blocked| X["Error returned, model not called"]
    B -->|Allowed| C["Model generates"]
    C --> D["Output checks: harm categories, protected material, groundedness"]
    D -->|Blocked| Y["Completion withheld, finish reason content_filter"]
    D -->|Allowed| E["Response returned with annotations"]

Which checks run depends on what you enabled and whether each control is set to block or only annotate. Some controls apply only to certain request paths, such as agents, so test the exact path you ship.

Annotations from these checks come back with the response, so you can log severities even when nothing was blocked, which is handy for tuning thresholds. Streaming responses are checked as they are produced, so a block can appear partway through the output.

In block (filter) mode, flagged content is returned as information and also stopped. In annotate mode, the service only reports what it found and lets the content through.

Annotations can be switched on even for severity levels that you've turned off for blocking, which makes annotate useful for tuning: you watch real traffic, see what would have been flagged, and then set thresholds with evidence rather than guesses.

"content_filter_results": {
  "violence": {"filtered": false, "severity": "low"},
  "jailbreak": {"filtered": false, "detected": false}
}

A common rollout is annotate first in staging, then block in production once false positives are understood.

Whichever mode you pick, store the annotations somewhere searchable. They are your evidence when a user asks why a request was refused. Severity thresholds are also a product decision: a clinical or security tool may need different limits from a children's app.

A direct attack (jailbreak) comes from the user typing instructions meant to override the system prompt. An indirect attack hides instructions inside content the model reads, such as a web page, email, document or tool result.

Agents that call tools make indirect attacks more serious, because a poisoned page can trigger an action. Prompt Shields has a control for each type.

Keyless access through Microsoft Entra ID removes secrets from the picture. There is nothing to leak into source control, nothing to rotate on a schedule, and every call is tied to a specific identity you can audit and restrict with RBAC.

In Azure-hosted apps, give the app a managed identity, assign it Foundry User on the resource, and authenticate with DefaultAzureCredential. Locally the same code uses your developer sign-in.

To enforce it, disable key-based access on the resource (the disableLocalAuth property) and back it with Azure Policy so nobody quietly re-enables keys. The trade-off is a little more setup for role assignments, and some older tools may still expect a key.

Start by separating the two. A 401 means the service didn't accept your identity. A 403 means it knew who you were but you aren't allowed to do that.

1. Confirm the endpoint is the project URL, not just the resource URL, and that the project name is spelled correctly.
2. For 401, check the token: expired, issued for the wrong tenant, or requested for the wrong audience. Confirm which identity DefaultAzureCredential actually picked with az account show.
3. For 403, list assignments with az role assignment list --assignee <id> --scope <resource-id> and look for Foundry User at the resource or project scope.
4. Allow a few minutes after a new role assignment, because propagation is not instant.
5. Check networking: with public access disabled, the caller must reach the private endpoint, or the firewall must allow its IP.
6. If key auth is disabled on the resource, a request using an API key will fail by design.

A 429 means you exceeded the rate limit on that deployment, usually tokens-per-minute or requests-per-minute. First read the Retry-After header, then work out whether it's a burst or a sustained shortfall.

1. Add retries with exponential backoff and jitter, honoring Retry-After.
2. Check the deployment's quota in the portal and compare it with real traffic.
3. Lower max_tokens where you can, since requested output size counts toward rate estimation.
4. Request more quota, or spread load across several deployments or regions.
5. Move steady heavy traffic to Provisioned, and push non-interactive work to a Batch deployment.

for attempt in range(5):
    try:
        return client.responses.create(model="gpt4o-prod", input=prompt)
    except RateLimitError as e:
        time.sleep(min(30, 2 ** attempt + random.random()))

Also separate 429 from 503 and timeouts. A 429 is a limit you control through quota and capacity, while a 503 points to service-side trouble where retries and a second region help more than extra quota.

If nobody is waiting for the answer, use a Batch deployment. Global Batch and Data Zone Batch run jobs asynchronously at about 50% lower cost with a 24-hour turnaround target, and they use a separate token quota, so a big job won't starve your online traffic.

Good fits are classification of historic data, bulk summarization, embedding backfills and offline evaluation runs.

1. Write requests as a JSONL file.
2. Upload it and create a batch job against a batch deployment.
3. Poll for completion and download the output file.

Pick Data Zone Batch instead of Global Batch when the data must stay in a US or EU zone. Don't use batch for chat or anything with a latency expectation.

Choose Provisioned when your traffic is steady, high and latency-sensitive, and when a 429 during peak hours is a business problem rather than an inconvenience.

The deciding signals are:

• a predictable baseline that keeps the PTUs well utilized
• a contractual or product latency target that variable shared capacity can't promise
• a need to budget a fixed monthly cost

Stay on Standard for prototypes, spiky or seasonal use, and low volume, since idle PTUs still cost money. Azure reservations can reduce the price of long-running PTU commitments. Many teams run Provisioned for the baseline and keep Standard for overflow experiments.

Use it to evaluate a fine-tuned model cheaply, before you commit to a production deployment type. It bills per token, runs in any region, and has no hourly hosting cost.

The catch is what you give up: no SLA and no data residency guarantee. So it's the wrong place for customer traffic or regulated data.

1. Fine-tune the model.
2. Deploy it with the Developer type and run your evaluation set.
3. If results hold, redeploy on Standard or Provisioned for production.

Before leaving Developer, record the metrics you saw. A fine-tuned model that looks good in evaluation can still behave differently under production sampling settings, so rerun a smaller check after redeploying.

You run evaluators against test conversations and look at the scores per agent version. Foundry offers agent-specific evaluators in addition to the usual quality ones.

Build a dataset of realistic and adversarial queries, run it from the portal or locally through the SDK, and compare versions before publishing. In production, continuous evaluation on sampled live traffic catches drift. Scores from an LLM judge are signals, so spot-check some rows by hand.

Foundry builds on OpenTelemetry. You connect an Application Insights resource to the project, instrument your app, and each model call, tool call and agent step is recorded as a span in a trace you can open in the portal.

import os
from azure.monitor.opentelemetry import configure_azure_monitor

configure_azure_monitor(
    connection_string=os.environ["APPLICATIONINSIGHTS_CONNECTION_STRING"]
)

A trace shows the order of steps, latency per step, token counts and errors, which is what you need to debug why an agent picked the wrong tool or why a run was slow.

Prompt and response content may be sensitive, so recording of message content is typically an explicit opt-in. Decide that on purpose rather than by default.

Add your own spans around business steps, such as validating an order or reranking results, so the trace tells the whole story rather than only the model calls.

The aim is that traffic to the resource and everything behind it never crosses the public internet.

1. Create a private endpoint for the Foundry resource in your VNet and set up private DNS.
2. Turn off public network access on the resource.
3. Put connected services, such as Storage, Azure AI Search and Cosmos DB, behind private endpoints too.
4. For agents, use the standard setup where you bring your own Storage, Cosmos DB and AI Search so data lands in resources you control.
5. Combine with managed identity and RBAC, and enforce it with Azure Policy.

Private networking is available for prompt and workflow agents, while hosted agents have their own network controls, so check the current docs for the agent type you use. DNS is the usual culprit when something works from the portal but not from the VNet.

You describe your function to the model with a JSON schema. The model never runs it. It returns a function call with arguments, your code executes the real function, and you send the result back so the model can finish the answer.

tools = [{"type": "function", "name": "get_order_status",
  "description": "Look up an order by id",
  "parameters": {"type": "object",
    "properties": {"order_id": {"type": "string"}},
    "required": ["order_id"]}}]

resp = openai.responses.create(model="gpt4o-prod", input="Where is order 8812?", tools=tools)
for item in resp.output:
    if item.type == "function_call":
        result = get_order_status(**json.loads(item.arguments))
        resp = openai.responses.create(
            model="gpt4o-prod", previous_response_id=resp.id,
            input=[{"type": "function_call_output", "call_id": item.call_id,
                    "output": json.dumps(result)}])

Write precise descriptions and validate arguments before running anything with side effects.

Two details trip people up. Arguments arrive as a JSON string you must parse, and the model can occasionally produce arguments that don't match your schema, so validate them first. For longer chains, loop until the response has no more function calls, and require explicit confirmation for functions that change data.

Model Context Protocol (MCP) gives agents a standard way to use external tools. You attach an MCP tool to the agent with the server's URL and a label, and the agent discovers the tools that server exposes.

from azure.ai.projects.models import MCPTool

tool = MCPTool(
    server_label="knowledge-base",
    server_url="https://<search>.search.windows.net/knowledgebases/<kb>/mcp",
    require_approval="always",
)

Two settings deserve care. Approval lets you require human or app approval before a tool runs, which matters for tools that change data. Authentication can go through a project connection, such as one using the project's managed identity.

A real example is Foundry IQ: a knowledge base is exposed over MCP, and agents call its retrieval tool to get grounded answers. Treat third-party servers as untrusted input, since their output can carry indirect prompt attacks.

Memory lets an agent remember things across sessions, such as a user's preferred language or earlier decisions, rather than only within one conversation. It is a separate store, not just a longer chat history.

Conceptually it works in three moves: facts are extracted from conversations, consolidated so duplicates and changes are merged, and retrieved when relevant to a new request. The agent then sees them as extra context.

• Scope memory per user so people never see each other's data.
• Decide what is allowed to be stored, and how it can be viewed or deleted.
• Remember it is in preview at the time of writing, so verify behavior in current docs.

A workflow agent is a declarative flow that coordinates a sequence of actions or several agents. Instead of one agent juggling everything, each step has one job, and the workflow decides who runs next.

Documented patterns include sequential (agent A feeds agent B), group chat (several agents discuss and converge) and human-in-the-loop (the flow pauses for a person to approve or supply input). You can design them visually in the portal or define them in code.

Use a workflow when the process is known and repeatable, such as research, then draft, then review. For open-ended tasks one well-equipped agent is often simpler. Workflow agents were in preview at the time of writing.

The model router is a deployable model that acts as one endpoint in front of several underlying models. For each prompt it picks a model based on factors such as complexity, cost and quality, so easy requests can go to a cheaper model while hard ones go to a stronger one.

You deploy it once and call it like any other deployment. The response tells you which underlying model answered, which helps with cost analysis and debugging.

Test it against your own evaluation set. If you need identical behavior every time, pin a specific model instead.

Fine-tuning adapts a supported base model with your own examples. The model must support it, so check its card first.

1. Prepare training (and validation) data as JSONL in the chat format, with consistent, high-quality examples.
2. Upload the files and create a fine-tuning job in the portal or via the SDK.
3. Watch training and validation loss to spot overfitting.
4. Deploy the result, using Developer deployment to evaluate it cheaply.
5. Compare it with the base model on your evaluation set, then promote it to a production deployment type.

A few hundred clean examples often beat thousands of noisy ones. Fine-tune for style, format and narrow behavior, not to inject facts that change.

Fine-tuning changes how a model behaves, but it is a poor way to teach it facts. Facts baked into weights go stale, can't be cited, and need another training run to update. RAG keeps knowledge outside the model and retrieves it at question time.

They are not exclusive. Plenty of systems fine-tune for output format and use RAG for the knowledge.

Treat it as a move to a new home rather than an in-place switch, and plan it in stages.

1. Inventory what you have: model deployments, connections, agents, files, evaluations and role assignments.
2. Create (or upgrade to) a Foundry resource and a Foundry project, with the networking and identity you want.
3. Redeploy models and recreate connections, preferring managed identity.
4. Rebuild agents on Agent Service and port code to the current azure-ai-projects 2.x conversation and response model.
5. Re-run evaluations side by side, then cut traffic over and retire the hub.

The new portal shows only Foundry projects, while the classic portal still shows hubs, so keep both in mind during the transition. Confirm any automated migration options in the current docs before relying on them.

Keep each concern in its own layer and make the data path private and identity-based.

flowchart LR
    U["Client app"] --> A["Published agent application"]
    A --> G[Guardrail]
    G --> M["Model deployment"]
    A --> K["Foundry IQ knowledge base"]
    K --> S["Azure AI Search"]
    S --> D[(Blob / SharePoint / OneLake)]
    A --> T["Tracing to Application Insights"]

1. A prompt agent holds instructions and the knowledge base tool, versioned and published.
2. A Foundry IQ knowledge base does retrieval with citations and user permissions.
3. Guardrails include Prompt Shields, because retrieved content is untrusted.
4. Apps and agents use managed identity, with private endpoints for the resource and its dependencies.
5. Evaluators run on a fixed test set before each release, with tracing and continuous evaluation afterwards.

Pick Provisioned throughput only if the baseline traffic justifies it.

Plan for the failure modes: stale index content, empty retrieval, a tool timeout and quota exhaustion. Tell the agent to say it doesn't know when retrieval returns nothing, add timeouts and retries around tools, and alert on groundedness drops rather than only on errors.

The two cases look different to your code.

• Blocked prompt: the request fails with HTTP 400 and an error code of content_filter. The model is never called. Details sit under prompt_filter_results.
• Blocked completion: the call succeeds with HTTP 200, but the choice's finish_reason is content_filter and the content is withheld or cut short. Details sit under choices[].content_filter_results.

if choice.finish_reason == "content_filter":
    log(choice.content_filter_results)
    return "I can't help with that request."

With streaming, content can be cut off mid-response. If legitimate users are being blocked, don't just switch filters off: put a test set through annotate mode, look at which category and severity fired, then adjust that threshold or create a narrower custom guardrail.

Log the category, severity and request id for every block. When you tune a guardrail or contact support, those details help far more than a screenshot of the refusal message.