How to Use Azure AI Agent Service: A Practical Walkthrough

Why Building Agents from Scratch Is Stalling Business Automation

Your team wants AI agents that actually close tickets, file reports, and move data between systems — but wiring a model to tools, memory, and enterprise apps is where automation projects stall. The plumbing eats the roadmap.

Microsoft's answer is Azure AI Agent Service, a fully managed agent platform announced at Ignite 2024 that removes the infrastructure layer entirely. You bring a model, instructions, and tools; Microsoft runs the compute, storage, and identity. At launch, it already sat inside an Azure AI footprint used by 60,000+ organizations.

One naming note before we go further: sources variously call this "Azure AI Agent Service," "Azure AI Foundry Agent Service," and "Foundry Agent Service." They refer to the same product family. Microsoft has folded the service into the Azure AI Foundry brand, so older announcements and current docs use different labels — but the capabilities described here are the same.

Why does "build it yourself" stall so often? Because an agent is not just a prompt. A production-grade agent needs several moving parts working together:

• A model layer you can swap without rewriting agent code — the service supports GPT-4o, Llama, DeepSeek and other catalog models
• A tool layer that reaches real business systems — 1,400+ Azure Logic Apps connectors cover Teams, Excel, Dynamics 365, Jira, Gmail, SAP, Stripe, ServiceNow and more
• Memory and knowledge connectors so the agent retains context across turns
• Identity and governance — Entra-based keyless auth, content filters, Prompt Shields, and VNet isolation
• Observability — OpenTelemetry-based tracing so you can see what the agent actually did

Hand-rolling each of those is a multi-quarter effort for most teams. Microsoft's own research with hundreds of organizations identified exactly these ingredients as the gap between a demo and a deployable agent.

The managed approach also changes the security calculus. When agents can act on web pages and enterprise systems, every new capability is a new attack surface — published research finds roughly 26% of agent skills carry at least one vulnerability. That is why independent verification matters as much as capability. Foundry's Toolboxes pattern — curate tools once, share via a managed MCP endpoint — points the same direction SkillzBench is building: a free directory of agent skills, every one pinned to a commit SHA, license-checked, and security-scanned before it ever reaches your agent.

ctaText: Skills, proven. Agent skills can automate the rest of this workflow — but not all skills are safe or useful. SkillzBench is building the free, independent directory that verifies, security-scans, and benchmarks every skill so you don't have to guess. Join the waitlist and be first to browse, test, and load skills into any agent. socialProofText: Built by AIQ Labs for agent builders, developers, and AI-curious teams who want verified skills — not vibes. Free forever: directory, MCP connector, benchmarks, and alerts.

The Three-Part Recipe: Model, Instructions, and Tools

Every agent you build on Azure AI Agent Service comes down to the same three ingredients: a model, a set of instructions, and the tools the agent is allowed to use. Strip away the enterprise packaging and that's the whole recipe — and per Microsoft's own guidance, it can be executed with just a few lines of code via the OpenAI or Azure AI Foundry SDKs.

Step one: pick your model. Foundry Agent Service works with GPT-4o, Meta Llama, DeepSeek and other models from the Foundry catalog, and Microsoft's documentation notes that models can be swapped without changing agent code source. That portability matters more than it sounds — you can start with a frontier model to nail the behavior, then trade down for cost once the instructions are stable.

Step two: write the instructions. This is the agent's job description: what it does, how it responds, when it should ask for clarification instead of guessing. Instructions are plain text, which is why non-technical team members can often draft and iterate on them directly.

Step three: attach tools. Tools are where automation actually happens. The launch announcement highlighted 1,400+ Azure Logic Apps connectors — Teams, Excel, Dynamics 365, Jira, Gmail, SAP, Stripe, ServiceNow and more — plus Azure Functions, Code Interpreter, and OpenAPI-specified tools with Entra ID authentication source.

Microsoft's documentation breaks the build path into three agent types:

• Prompt agents — configuration only, no code; the fastest way to a working agent.
• Voice agents — currently in preview, supporting real-time spoken conversation.
• Hosted agents — bring your own code as a container, with a managed endpoint, autoscaling, and Entra identity.

Microsoft's on-ramp advice is refreshingly concrete: "Building your first agent? Start with a prompt agent using either the Foundry portal, or the quickstart for creating a prompt agent with code" source. Non-technical readers should take the portal route; developers will want the code quickstart so the agent definition lives in version control and deploys through CI/CD like any other infrastructure.

The same curation principle applies beyond Azure. Agent skills — packaged instruction files that teach coding agents and chat agents new workflows — can extend what your agent does, but published research has found roughly 26% of skills carry at least one vulnerability. SkillzBench is building a free, independent directory that security-scans and benchmarks every skill before you load it, so your agent only ever sees what you've verified.

Step-by-Step: Automating Real Work with Tools and Connectors

An agent that can only chat is a demo. An agent that can click, email, and file tickets is automation — and that's where Azure AI Agent Service earns its keep.

The service's tool layer is broad. Through Azure Logic Apps, agents get access to 1,400+ connectors spanning Microsoft Teams, Excel, Dynamics 365, Jira, Gmail, SAP, Stripe, ServiceNow, and more. Azure Functions handle custom code actions, Code Interpreter runs Python, and OpenAPI 3.0-specified tools bring your own APIs with Entra ID authentication. In practice, that means one agent can research a topic, generate a deck via Functions, and email the result through Logic Apps — Microsoft's own showcase of multi-step composability.

For a fully worked example, the Browser Automation Tool (Preview) is the most complete walkthrough Microsoft publishes. The setup sequence:

• Provision a Playwright Workspace and generate an access token (Python 3.8+ required)
• Assign the Contributor RBAC role on the workspace to your project identity
• Create a "Serverless Model" connection in the Foundry Management Center, targeting the wss:// URI for your Playwright workspace
• Create the agent with the browser_automation tool config, then follow the SDK pattern: create an agent, open a thread, post a message, run it, inspect run_steps, and clean up

Unlike pixel-based RPA bots, this tool reads the page's DOM and accessibility tree, letting the agent reason about elements by role and label — Microsoft's framing, not an independent benchmark, so treat the resilience claims with appropriate skepticism.

Before you build with it, read Microsoft's warning. The company states plainly that the tool "comes with significant security risks": AI judgment errors plus malicious or confusing instructions embedded in web pages can cause the agent to execute commands you never intended, potentially compromising browsers, accounts, and enterprise systems. Microsoft's recommendation is to run it only on low-privilege virtual machines with no access to sensitive data. That's not boilerplate — it's the platform vendor telling you the attack surface is real.

This is the same risk pattern showing up across the agent ecosystem. Published research has found that roughly 26% of agent skills carry at least one vulnerability, which is why independent security scanning and verification matter before anything gets wired into your agent. It's also why SkillzBench is building a free directory of agent skills where every skill is pinned to a commit SHA, license-checked, and security-scanned before you ever load it — because the connectors Microsoft ships are audited; the skills your team downloads from the internet usually aren't.

Wire up the tools, scope the permissions tightly, and test each capability before it touches production data.

Governance, Costs, and Composability: Keeping Agents Under Control

An agent that can act is an agent that can misbehave — which is why governance and cost controls belong in your build plan from day one, not as an afterthought.

Foundry Agent Service ships with a set of enterprise controls designed for exactly this. You can bring your own storage and your own VNet with no public egress, authenticate through Entra ID with keyless or on-behalf-of credentials, and trace every agent run via OpenTelemetry. On the model side, Prompt Shields mitigate cross-prompt injection attacks (XPIA) and content filters screen inputs and outputs. These aren't optional add-ons; they're the reason the service is positioned for regulated, enterprise-scale deployments.

The need for those controls is real, not theoretical. Microsoft's own documentation on the Browser Automation Tool warns that "errors in judgment by the AI and the presence of malicious or confusing instructions on web pages" can cause agents to execute unintended commands, and recommends running browser automation on low-privilege virtual machines with no access to sensitive data. The same caution applies beyond Microsoft's tools: published research has found roughly 26% of agent skills carry at least one vulnerability — a reminder that anything you load into an agent deserves independent security scanning.

Then there's the bill. Deep Research, the service's flagship research feature, runs on the o3-deep-research model at $10 per 1M input tokens and $40 per 1M output tokens (cached input drops to $2.50 per 1M), with separate charges for Grounding with Bing Search and the base GPT model used for clarifying questions. More broadly, prompt and hosted agents bill per-call inference plus tool usage, with hosted agents adding container compute costs. Voice agents layer on Voice Live, speech recognition and synthesis, and potentially telephony charges. Before scaling an automation, model the token math — output-heavy research tasks are where costs surprise teams.

Once controls and budgets are in place, the payoff is composability. Microsoft's showcase pattern chains three agents: one performs deep web research, a second generates a slide deck via Azure Functions, and a third emails the result to decision makers through Azure Logic Apps. Research stops being a one-off task and becomes a reusable building block in a multi-step workflow.

Foundry also formalizes tool curation through its Toolboxes pattern: teams curate tools once — web search, file search, code interpreter, MCP servers, custom functions — and share them across agents via a single managed MCP endpoint with centralized authentication, governance, and versioning. That's the right instinct, and it's one the wider agent ecosystem is converging on. SkillzBench is building on the same idea for agent skills: a free, independent directory where every SKILL.md package is pinned to a commit SHA, license-checked, security-scanned, and benchmarked for real lift, delivered through a free MCP connector that shows your agent only the skills you choose. If you want curated, verified capabilities for your agents without the guesswork, you can join the waitlist now.

Governed well, agents stop being a risk you tolerate and become an automation capability you trust.

Going Further: Extending Your Agent with Verified Skills

Your Azure AI agent is running, but it likely lacks the specific, verified workflows needed for true automation. This is where agent skills come in. These are SKILL.md instruction packages designed for tools like Claude Code, Codex, and Cursor that automate complex tasks. However, the ecosystem is currently unregulated. Published research indicates that approximately 26% of these skills carry at least one security vulnerability, making blind adoption risky.

Microsoft’s own architecture validates the need for a curation layer. The Azure AI Foundry Agent Service introduces "Toolboxes," which allow teams to curate tools once and share them via a managed MCP endpoint with centralized authentication. This pattern ensures that agents only access approved, versioned capabilities. Microsoft documentation highlights that this approach simplifies governance while enabling multi-agent workflows. It shifts the focus from raw model power to structured, verifiable tool usage.

SkillzBench is building a free, independent directory that applies this same rigorous curation logic to the broader agent-skills ecosystem. Unlike open repositories, SkillzBench pins every skill to a specific commit SHA to ensure version integrity. It then performs independent security scans, license checks, and quality judgments before a skill is listed. This process generates a 0–100 Trust score, providing a clear, objective metric for security and reliability.

The platform also focuses on actual utility through benchmarking. Every skill is tested in a sandbox environment with and without the skill enabled. The difference is published as a "Lift" metric, showing exactly how much performance improvement the skill provides. This data is kept separate from community ratings, ensuring that monetary incentives or popularity never influence the technical assessment.

For developers and teams integrating these skills, the verification pillars include:

• Continuous discovery and deduplication of skills from major platforms like GitHub.
• Automated security scanning and license validation to mitigate supply chain risks.
• Sandboxed benchmarking to measure real-world performance impact via the Lift metric.
• A free MCP connector that allows agents to load only the specific, verified skills you select.

This approach aligns directly with the enterprise-grade governance models seen in Azure AI Foundry. It ensures that your agents operate with verified, safe, and effective instructions. SkillzBench is currently in pre-launch, but the infrastructure for verification and benchmarking is being built now.

Skills, proven. Agent skills can automate the rest of your workflow, but you should not guess which ones are safe. Join the SkillzBench waitlist to be notified when the free directory and MCP connector go live.