AI-103 Evaluation and Observability: How to Test, Trace and Safety-Check Microsoft Foundry Apps in 2026
The evaluation and observability half of AI-103, covered properly. Every built-in Microsoft Foundry evaluator, portal and SDK evaluation labs, OpenTelemetry tracing setup, and the content-filter versus evaluator distinction the exam hinges on.
Examinotion Team

Last updated: August 2026. Written and fact-checked by the Examinotion editorial team against the official Microsoft Learn AI-103 skills outline and the Microsoft Foundry product documentation.
TL;DR AI-103 no longer stops at building an AI app. It tests whether you can prove one works. Microsoft Foundry supplies built-in evaluators for quality, retrieval, agent behaviour and safety, plus OpenTelemetry tracing for latency, tokens and tool calls. Together these span the exam's two heaviest domains, worth 55 to 65 per cent.
Most candidates arrive at AI-103, the exam behind the Microsoft Certified: Azure AI Apps and Agents Developer Associate credential, able to build things. They can wire a retrieval-augmented generation pipeline, deploy a model, and get an agent to call a tool. Then they hit a block of questions that ask something entirely different: how would you know it is working, and how would you prove it is safe?
That block is bigger than most people expect, and it is the single most common blind spot we see. This guide closes it. You will learn where evaluation and observability sit in the skills outline, which evaluators Microsoft actually ships, how to run them from both the portal and the SDK, how to turn on tracing, and how to tell an evaluator question from a content-filter question when the two look almost identical on the screen.
Why AI-103 now tests whether you can prove your AI works
The AI-103 skills outline was rewritten with an effective date of 16 April 2026, and the study guide page was last updated on 7 July 2026 [1]. The rewrite pushed measurement, safety instrumentation and production monitoring from a footnote into the core of the exam.
Microsoft's own bullet is unambiguous about the expectation:
Set up observability by implementing tracing, token analytics, safety signals, and latency breakdowns.
That line sits inside "Implement generative AI and agentic solutions", the largest domain on the paper [1]. It is not a peripheral topic.
The reason is straightforward. An organisation shipping a generative AI feature to customers carries a risk profile that a traditional CRUD application does not. It can fabricate. It can be steered off-task by a hostile input. It can quietly degrade as the underlying model or the source data changes. Microsoft has built tooling for all three, and the exam expects a developer to know that tooling exists and how to reach for it.
Where evaluation and observability sit in the skills measured
The AI-103 exam is organised into five domains [1]. Retrieval-augmented generation is abbreviated as RAG throughout this article.
| Domain | Weighting |
|---|---|
| Plan and manage an Azure AI solution | 25-30% |
| Implement generative AI and agentic solutions | 30-35% |
| Implement computer vision solutions | 10-15% |
| Implement text analysis solutions | 10-15% |
| Implement information extraction solutions | 10-15% |
Evaluation and observability content is concentrated in the top two rows, which together account for 55 to 65 per cent of the exam. The specific bullets, quoted verbatim from the skills outline [1], are these.
Under Plan and manage an Azure AI solution, in the "Manage, monitor, and secure AI systems" group:
Monitor model performance, drift, safety events, and grounding quality
And in the "Implement responsible AI across generative AI and agentic systems" group, which is the densest cluster of relevant material on the whole outline:
Configure safety filters, guardrails, risk detection, and content moderation" "Apply responsible AI instrumentation, including evaluators, safety evaluations, and explanation tooling" "Implement auditing through trace logging, provenance metadata, and approval workflows" "Govern agent behavior with oversight modes, constraints, and tool-access controls
Under Implement generative AI and agentic solutions:
Evaluate models and apps, including detecting fabrications, relevance, quality, and safety" "Integrate monitoring into deployed agents, evaluate agent behavior, and perform error analysis
Those bullets use American spelling because they are Microsoft's exact exam text and we have not altered them.
Read that list again and notice how many distinct verbs appear: configure, apply, implement, evaluate, integrate, monitor, govern. This is implementation-level material, not a conceptual survey. If your preparation so far has been the six Responsible AI principles at a philosophical level, you have covered the ethics but not the engineering. Our guide to Microsoft's Responsible AI principles covers the conceptual layer; this article is the layer beneath it.
Before you start: the terminology trap that will cost you marks
Microsoft renamed Azure AI Foundry to Microsoft Foundry, and the documentation now lives under learn.microsoft.com/azure/foundry/. Every citation in this article uses the current name. If your study notes, a video course, or an AI assistant still says "Azure AI Foundry", the substance may be right but the naming is a year out of date.
There is a second, more urgent trap. The Azure OpenAI Assistants API is deprecated, and Microsoft's documentation states the retirement date plainly [12]:
The Assistants API is deprecated and will be retired on August 26, 2026. Use the generally available Microsoft Foundry Agents service.
That is weeks away from the time of writing. Any tutorial, course or generated code sample that builds an agent using Assistants, Threads and Runs is teaching a dead-end API. The successor architecture is the Foundry Agents service alongside the Responses API. If you are working through older AI-103 material, check the agent-building sections first, because they are the ones most likely to be stale.
A third change affects code specifically. Most existing tutorials install the standalone azure-ai-evaluation package and call a top-level evaluate() function. Microsoft's current cloud-evaluation how-to, updated 22 July 2026, instead uses azure-ai-projects version 2.2.0 or later with an OpenAI-evals-compatible interface [7]. Follow an older tutorial verbatim today and you will hit import errors. Learn the current path.
Prerequisites: project, roles and a judge model
Three things must be in place before an evaluation will run.
A Microsoft Foundry project. Evaluations are scoped to a project, and results are stored against it.
The right role. Microsoft's portal how-to lists the requirement as the Foundry User role on the Foundry project [6]. Note that these role names changed recently. The RBAC documentation confirms the rename and, importantly, tells you what to do about it [10]:
Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously named Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. You might still see the previous names in some places while the rename rolls out. The role IDs and core permissions are unchanged by the rename.
The practical lesson for infrastructure-as-code, and a plausible exam distractor, is to reference role IDs rather than display names while a rename is mid-rollout.
A judge model, but only for some evaluators. This is the distinction candidates most often miss. AI-assisted evaluators use a large language model as a judge and therefore need a deployed model, for example gpt-4.1-mini, supplied through an Azure OpenAI connection [6]. Risk and safety evaluators do not. Microsoft is explicit [5]:
Risk and safety evaluators use the hosted Foundry Evaluation service and don't require a deployment_name initialization parameter. Unlike LLM-as-judge evaluators such as coherence and fluency, these evaluators run against Microsoft's hosted safety models.
Memorise that split. A question that asks why a safety evaluation succeeded in a project with no model deployment is testing exactly this.
The evaluator catalogue, grouped the way the exam thinks about it
Microsoft's built-in evaluator reference groups evaluators into families [2]. Learning the families matters more than memorising every name, because exam scenarios describe a problem and expect you to reach for the right family.
General-purpose quality evaluators
Coherence measures logical consistency and flow. Fluency measures natural language quality and readability. Both are AI-assisted, both return a 1 to 5 score [2].
Textual similarity evaluators
F1 Score, BLEU, GLEU, ROUGE and METEOR compare a response against a ground-truth answer using n-gram overlap or token-level precision and recall. These are mathematical, not AI-assisted, so they need no judge model. Similarity is the exception in this group and is AI-assisted [2].
These are the evaluators to reach for when a scenario mentions translation, summarisation against a reference, or any task with a single known-correct answer.
RAG and retrieval evaluators
This family is where most AI-103 evaluation questions live, because RAG is the dominant architecture on the exam [3].
| Evaluator | What it measures | Output | Needs a judge model |
|---|---|---|---|
| Retrieval | How relevant the retrieved context chunks are to the query | 1-5, threshold 3 | Yes |
| Document Retrieval | Fidelity, NDCG, XDCG, Max Relevance and Holes against human-labelled relevance judgements | Composite metrics | No |
| Groundedness | Whether the response stays within the supplied context without fabricating | 1-5, threshold 3 | Yes |
| Groundedness Pro (preview) | Strict consistency with context, via Azure AI Content Safety | True or False | No |
| Relevance | Accuracy, completeness and directness of the response to the query | 1-5, threshold 3 | Yes |
| Response Completeness (preview) | How completely the response covers the expected information | 1-5, threshold 3 | Yes |
The pairing you must be able to explain is Groundedness against Response Completeness. Microsoft draws the line precisely [3]:
Groundedness focuses on the precision aspect of the response. It doesn't contain content outside of the grounding context. Response completeness focuses on the recall aspect of the response. It doesn't miss critical information compared to the expected response or ground truth.
Precision and recall. A response can be perfectly grounded and still useless because it omitted half the answer. A response can be complete and still dangerous because it invented a supporting detail. You need both metrics to describe a RAG system honestly.
Do not conflate Groundedness with Groundedness Pro either. They are architecturally different, not two maturity levels of the same thing. Groundedness is an AI-assisted evaluator returning a graded score. Groundedness Pro calls the Azure AI Content Safety service and returns a binary verdict with no model deployment required [3].
If you built the pipeline in our RAG on Azure AI Search walkthrough, this family is how you would now measure it.
Agent evaluators
Foundry splits agent evaluation into system evaluation, which asks whether the outcome was right, and process evaluation, which asks whether the steps were right [4]. That split is a gift to anyone structuring revision notes.
Microsoft frames the whole family with an analogy worth remembering [4]:
Foundry provides built-in agent evaluators that function like unit tests for agentic systems-they take agent messages as input and output binary Pass/Fail scores (or scaled scores converted to binary scores based on thresholds).
System evaluation covers Task Completion, Customer Satisfaction, Task Adherence, Task Navigation Efficiency and Intent Resolution. Intent Resolution asks whether the agent correctly identified what the user wanted. Task Adherence asks whether the agent's actions respected its rules, procedures and policy constraints [4].
Process evaluation covers the tool-call layer: Tool Call Accuracy, Tool Selection, Tool Input Accuracy, Tool Output Utilization and Tool Call Success. Tool Input Accuracy alone is validated against six criteria, described by Microsoft as "groundedness, type compliance, format compliance, required parameters, no unexpected parameters, and value appropriateness" [4].
The exam-relevant insight is diagnostic. If an agent gives a wrong final answer, system evaluators tell you that it failed and process evaluators tell you where. Tool Selection failing points at the agent's reasoning about which tool to use. Tool Input Accuracy failing points at parameter construction. Tool Output Utilization failing means the agent retrieved the right data and then ignored it. That is error analysis, which is a skills-outline bullet in its own right.
Most of these agent evaluators are still in preview. Microsoft's study guide notes that the exam concentrates on generally available features but may include preview features where they are commonly used, so know both status and function.
Risk and safety evaluators
Eight risk and safety evaluators apply to both models and agents: Hateful and unfairness, Sexual, Violence, Self harm, Protected materials, Code vulnerability, Ungrounded attributes, and Indirect Attack, also known as cross-domain prompt injected attack or XPIA, which applies to models only. Two further evaluators are agent-only and in preview: Prohibited actions and Sensitive data leakage [5].
Two of these deserve extra attention because they surprise people. Code vulnerability scans generated code across Python, Java, C++, C#, Go, JavaScript and SQL for eighteen named flaw classes including SQL injection, path injection, hardcoded credentials and insecure randomness. Ungrounded attributes catches a subtler failure: the model inferring someone's demographics or emotional state from thin evidence [5].
The scoring model differs from every other family [5]:
Content safety evaluators use a 0-7 severity scale. Given a numerical threshold (default 3), the evaluator outputs pass if the score is less than or equal to the threshold, or fail otherwise.
Note the direction. For safety, lower is better and the threshold is a ceiling. For quality evaluators such as Relevance, higher is better and the threshold is a floor. Getting that inversion the wrong way round is an easy mark to lose.
Across a dataset, Foundry aggregates results into a defect rate, described as "the percentage of undesired content detected in the response from your AI system" [5]. Microsoft does not publish a more precise formula, so do not memorise arithmetic that has not been documented.
Microsoft's own recommended combinations
The built-in evaluator reference publishes starting recipes, which is as close to an official answer key as this topic offers [2]:
RAG applications: Retrieval + Groundedness + Relevance + Content Safety Agent applications: Tool Call Accuracy + Task Adherence + Intent Resolution + Rubric + Content Safety Translation applications: BLEU + METEOR + Fluency + Coherence All applications: Add risk and safety evaluators (Hate and Unfairness, Sexual, Violence, Self-Harm) for responsible AI practices
If you learn nothing else from this section, learn those four lines.
Lab 1: Run an evaluation from the portal, no code
The portal path is the fastest way to build intuition, and it is fair game for exam questions that describe a UI journey. Microsoft documents four entry points [6]:
Evaluation page: From the left pane, select Evaluation > Create. Models page: Go to your model, select the Evaluation tab, then select Create. Agents page: Go to your agent, select the Evaluation tab, then select Create. Agent playground: Go to your agent, select the Playground tab, then select Metrics > Run full evaluation.
The wizard then walks seven steps [6]: choose the evaluation target (Agent, Model, Dataset or Traces), choose the scope (full conversations or individual turns), choose the data source (simulated data, existing conversations, an existing dataset or existing traces), configure any prompt overrides, map dataset fields to evaluator inputs, select your testing criteria, then review and submit. A run reports as In Progress, then Completed, Partial or Failed.
Two options in that list are worth pausing on. Simulated data means you do not need a hand-built test set to get started, because Foundry can generate one. Traces means you can evaluate real production traffic that has already been captured in Application Insights, which is the bridge between the observability half of this article and the evaluation half.
Lab 2: Evaluation from the Python SDK
The dataset format comes first, because field names are exactly the kind of detail an exam question turns on. For individual-turn evaluations, Foundry expects these fields [6]: query and response are required; context, ground_truth, tool_calls and tool_definitions are optional. Multi-turn conversation evaluations use messages and tool_definitions instead. Only CSV and JSONL are accepted as uploaded datasets [6].
A single JSONL row looks like this [3]:
{"query": "What are the store hours?", "context": "Our store is open Monday-Friday 9am-6pm and Saturday 10am-4pm.", "response": "The store is open weekdays from 9am to 6pm and Saturdays from 10am to 4pm."}
Now the current SDK path [7]:
pip install "azure-ai-projects>=2.2.0"
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import TestingCriterionAzureAIEvaluator
from openai.types.evals.create_eval_jsonl_run_data_source_param import (
CreateEvalJSONLRunDataSourceParam, SourceFileID,
)
project_client = AIProjectClient(endpoint=endpoint, credential=DefaultAzureCredential())
openai_client = project_client.get_openai_client()
testing_criteria = [
TestingCriterionAzureAIEvaluator(
type="azure_ai_evaluator",
name="coherence",
evaluator_name="builtin.coherence",
initialization_parameters={"model": model_deployment_name},
data_mapping={"query": "{{item.query}}", "response": "{{item.response}}"},
),
]
eval_object = openai_client.evals.create(
name="dataset-evaluation",
data_source_config=data_source_config,
testing_criteria=testing_criteria,
)
eval_run = openai_client.evals.runs.create(
eval_id=eval_object.id,
name="dataset-run",
data_source=CreateEvalJSONLRunDataSourceParam(
type="jsonl", source=SourceFileID(type="file_id", id=data_id)
),
)
Three details carry exam weight. Evaluator names are namespaced as builtin.{name}, for example builtin.coherence or builtin.violence. Running several evaluators at once means appending more TestingCriterionAzureAIEvaluator entries to the same list, not making several separate runs. And the data_mapping mini-language distinguishes {{item.field}}, which pulls from a static dataset row, from {{sample.output_text}}, which pulls the live output of a model or agent being exercised during the run [7].
One more constraint that generates real-world bugs and plausible exam distractors: every evaluator in a run must support the same evaluation level, either turn or conversation. You cannot mix incompatible levels in a single run [2].
Lab 3: Turning on tracing
Tracing is the observability half of the skills bullet, and the documentation was updated on 1 August 2026, making it one of the freshest pages in the Foundry set [8].
Start with what tracing captures [8]:
At a high level, tracing captures:
- User inputs and agent outputs.
- Tool usage, including tool calls and results.
- Token consumption.
- Time signals such as duration and latency."
Line those four up against the skills bullet quoted at the top of this article: tracing, token analytics, safety signals, latency breakdowns. The mapping is nearly one to one, which is a strong hint about how the exam approaches the topic.
The underlying standard matters too [8]:
Foundry uses OpenTelemetry semantic conventions so traces are consistent across supported tools and integrations.
OpenTelemetry, usually shortened to OTel, is a vendor-neutral standard for collecting telemetry. Foundry exports traces to Azure Monitor Application Insights, which is the only backend named in the documentation. The vocabulary the exam may use is the standard OTel vocabulary: a trace is one end-to-end run, a span is one operation inside it, attributes are the key-value metadata on a span, and trace exporters move the data to a backend [8].
Availability is not blanket, and this is the sort of nuance that separates a pass from a near-miss [8]:
Tracing is generally available for prompt and hosted agents. Workflow and external agents are in preview.
How you enable tracing depends on what you built [9]. If you are using the Microsoft Agent Framework or Semantic Kernel, agents emit traces automatically once the Foundry project is connected to an Application Insights resource, with no extra code or packages. For LangChain and LangGraph, install the Microsoft OpenTelemetry distribution and enable it explicitly:
from microsoft.opentelemetry import use_microsoft_opentelemetry
use_microsoft_opentelemetry(
enable_azure_monitor=True,
sampling_ratio=1.0,
instrumentation_options={
"langchain": {"enabled": True, "agent_id": "...", "agent_name": "..."}
},
)
For the OpenAI Agents SDK, wire OpenTelemetry manually using opentelemetry-instrumentation-openai-agents and export with AzureMonitorTraceExporter.from_connection_string(conn) [9].
Three environment variables control whether prompt and response content is recorded at all: OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, OTEL_SEMCONV_STABILITY_OPT_IN and AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING [9]. Those exist because message content is often sensitive, which leads to a warning worth quoting in full [8]:
Don't store secrets, credentials, or tokens in prompts, tool arguments, or span attributes. Redact or minimize personal data and other sensitive content before it appears in telemetry. Treat trace data as production telemetry and apply the same access controls and retention policies you use for logs and metrics.
Traces appear in the portal under Observability, then Traces, typically within two to five minutes of execution [8]. Viewing them needs access to the connected Application Insights resource, starting with Log Analytics Reader, and Privileged Monitoring Data Reader where the underlying tables are protected. Configuring ingestion in the first place needs Contributor or higher on the Application Insights resource [8].
If you built an agent using our Microsoft Foundry agent walkthrough, enabling tracing on it is a twenty-minute exercise that will teach you more than an hour of reading.
Content filters versus evaluators: the distinction the exam hinges on
This is the single most important disambiguation in the topic, and it catches people because the two systems share vocabulary. Content filters classify hate, sexual, violence and self-harm content. So do the risk and safety evaluators. Same harm taxonomy, completely different mechanism.
| Content filters | Evaluators | |
|---|---|---|
| When they run | Inline, on every request, in real time | Offline or scheduled, on a sample |
| What they act on | The live prompt and completion | A dataset, a simulation, or historical traces |
| What they do | Block content before the user sees it | Score and report, changing nothing |
| Where configured | On the model deployment or agent | In the Evaluation service |
| Output | Filtered or allowed, per severity threshold | Scores, pass rates and a defect rate |
Content filters classify into four categories at four severity levels, "safe, low, medium, and high", for both text and image content [11]. The threshold behaviour includes one rule worth memorising verbatim [11]:
Content at the 'safe' level is always annotated but never blocked, regardless of threshold setting.
Beyond the four core harms, the filtering stack adds Prompt Shields for direct jailbreak attempts and indirect prompt injection, a protected material detector, and custom blocklists of your own terms.
A small terminology wrinkle to watch: the content-filter documentation labels the category "Hate and Fairness", while the evaluator is named "Hate and Unfairness". Same underlying risk, slightly different label depending on which surface you are looking at.
How to tell which one a question is asking about. If the scenario mentions blocking, real time, a live end user, or something happening at request time, it is a content filter. If it mentions testing before release, a dataset, a score, a pass rate, regression testing, or measuring across many past conversations, it is an evaluator. That heuristic resolves the great majority of these questions.
Continuous evaluation and what Microsoft has not documented
Foundry supports scheduling evaluations to run on a recurring basis, and continuous evaluation that automatically evaluates sampled agent responses in production. The Traces evaluation target then lets you evaluate real interactions already captured in Application Insights, which is post-deployment evaluation of genuine production traffic rather than a synthetic test set [6][7].
An honest caveat, because it matters for your revision: we could not find an official Microsoft reference pipeline showing evaluation wired into GitHub Actions or Azure DevOps with concrete YAML. The capability is documented conceptually; a canonical code sample is not. Understand the pattern, which is to run evaluations as a quality gate on every change to prompts, models or retrieval configuration, and treat any specific pipeline syntax you find in third-party material as illustrative rather than authoritative.
An honest word on difficulty
This is not an easy area, and pretending otherwise would not help you. The evaluator catalogue is large, a meaningful share of it is in preview and therefore still moving, the SDK surface changed direction in mid-2026, and the product itself was renamed inside the last year. Preparing from anything more than a few months old carries a real risk of learning the wrong names.
The compensation is that this material rewards hands-on time far more than reading. One afternoon spent running a portal evaluation against a RAG app you already built, then enabling tracing on an agent you already built, will fix more of this in memory than a week of notes. The concepts are not intellectually difficult. They are just unfamiliar, and unfamiliarity dissolves quickly with a keyboard in front of you.
One relief: AI-103 is an associate-level exam, and Microsoft permits access to Microsoft Learn during associate and expert exams [13]. You can look things up. The timer keeps running, so this rescues a half-remembered evaluator name, not a topic you never studied.
Frequently Asked Questions
What is the difference between an evaluator and a content filter in Microsoft Foundry?
A content filter runs inline on every request and blocks content in real time before a user sees it, based on a severity threshold. An evaluator runs offline or on a schedule against a dataset, a simulation or historical traces, and produces scores rather than blocking anything. They share harm categories but serve opposite purposes.
Which evaluators should I use for a RAG application?
Microsoft's published recommendation for RAG applications is Retrieval, Groundedness, Relevance and Content Safety together [2]. Retrieval scores whether the right context was fetched, Groundedness scores whether the answer stayed inside that context, and Relevance scores whether it addressed the question. Add Response Completeness when you have ground-truth answers available.
Do risk and safety evaluators need a deployed model?
No. Risk and safety evaluators run against Microsoft's hosted safety models through the Foundry Evaluation service, so they need no deployment_name parameter and no Azure OpenAI model deployment [5]. AI-assisted quality evaluators such as Coherence, Fluency, Relevance and Groundedness do require a judge model deployment.
How do you enable tracing for an agent in Microsoft Foundry?
Connect your Foundry project to an Application Insights resource first, because tracing collects nothing until that link exists. Agents built with Microsoft Agent Framework or Semantic Kernel then emit traces automatically with no extra code. LangChain, LangGraph and OpenAI Agents SDK agents need explicit OpenTelemetry instrumentation adding [9].
What is the passing score for AI-103 and can I use Microsoft Learn during the exam?
You need 700 out of 1000 to pass, and the exam runs for 120 minutes [14]. AI-103 is an associate-level exam, and Microsoft permits access to learn.microsoft.com during associate and expert exams, excluding Q&A, Practice Assessments and your profile. The exam timer continues while you browse [13].
Is the Assistants API still valid for building agents on AI-103?
No. Microsoft's documentation states the Assistants API is deprecated and retires on 26 August 2026, directing developers to the Microsoft Foundry Agents service instead [12]. Any study material teaching Assistants, Threads and Runs is out of date. Learn the Foundry Agents service and the Responses API instead.
How much of AI-103 is evaluation and observability?
There is no published percentage for the topic itself. The relevant bullets sit inside "Plan and manage an Azure AI solution" at 25 to 30 per cent and "Implement generative AI and agentic solutions" at 30 to 35 per cent [1]. Those two domains total 55 to 65 per cent, though not all of that is evaluation content.
Conclusion
Evaluation and observability turned AI-103 from an exam about building AI into an exam about operating it responsibly. The good news is that the surface area, while wide, is well documented and highly learnable. Know the evaluator families and what each one diagnoses. Know that safety evaluators score low-is-good on 0 to 7 while quality evaluators score high-is-good on 1 to 5. Know that content filters block and evaluators measure. Know that tracing needs an Application Insights connection before it captures anything at all.
Then practise the distinctions under exam conditions, because recognising which of two very similar things a question is describing is a skill in itself.
Ready to test yourself? Work through Examinotion's AI-103 practice tests to find the gaps while there is still time to close them, or read the full AI-103 study guide for domain-by-domain coverage. If you have not settled on an exam yet, compare the full catalogue on the Microsoft exams hub.
Further reading from Examinotion: How to Pass the AI-103 Exam, the AI-103 30-day study plan, AI-103 practice questions with worked answers, and Copilot Studio versus Microsoft Foundry if you are deciding which platform to build on.
Sources
- Study guide for Exam AI-103 - Microsoft Learn, skills measured effective 16 April 2026, page updated 7 July 2026, accessed 2026-08-05
- Built-in evaluators reference - Microsoft Foundry documentation, updated 2 June 2026, accessed 2026-08-05
- Retrieval-augmented generation (RAG) evaluators - Microsoft Foundry documentation, updated 2 June 2026, accessed 2026-08-05
- Agent evaluators for generative AI - Microsoft Foundry documentation, updated 2 June 2026, accessed 2026-08-05
- Risk and safety evaluators for generative AI - Microsoft Foundry documentation, updated 18 May 2026, accessed 2026-08-05
- Run evaluations from the Microsoft Foundry portal - Microsoft Foundry documentation, updated 19 June 2026, accessed 2026-08-05
- Run evaluations from the SDK - Microsoft Foundry documentation, updated 22 July 2026, accessed 2026-08-05
- Agent tracing overview - Microsoft Foundry documentation, updated 1 August 2026, accessed 2026-08-05
- Configure tracing for AI agent frameworks - Microsoft Foundry documentation, updated 26 June 2026, accessed 2026-08-05
- Role-based access control for Microsoft Foundry - Microsoft Foundry documentation, updated 13 July 2026, accessed 2026-08-05
- Harm categories and severity levels in Microsoft Foundry - Microsoft Foundry documentation, updated 3 August 2026, accessed 2026-08-05
- Azure OpenAI Assistants API concepts - Microsoft Learn, updated 24 June 2026, accessed 2026-08-05
- Exam duration and exam experience - Microsoft Learn, updated 4 June 2026, accessed 2026-08-05
- Microsoft Certified: Azure AI Apps and Agents Developer Associate - Microsoft Learn, updated 4 August 2026, accessed 2026-08-05
Related Articles

AB-730 Exam Update October 2026: The New Agents Domain, Cowork and Work IQ Explained
Microsoft is restructuring the AB-730 AI Business Professional exam on 20 October 2026, transitioning its official blueprint from foundational prompt engineering toward autonomous enterprise automation.

AB-650 Study Plan: 30 Days to the AI Services Administrator Exam (2026)
A realistic four-week study plan for Microsoft exam AB-650 (Administering Microsoft 365 and AI Services), mapping the three official skill domains to daily Microsoft Learn modules and hands-on tenant practice.

How to Book a Microsoft AI Certification Exam: Pearson VUE Step by Step (2026)
A step-by-step guide to booking any Microsoft AI certification exam in 2026, from creating your Microsoft Learn profile to scheduling with Pearson VUE, choosing online or test centre delivery, applying a voucher, and rescheduling.