|
Most AI outages I see aren't model problems. They're network problems. A DNS zone that isn't linked, a subnet that's too small, a firewall that silently drops an agent's outbound call. Here's a practical, step-by-step guide to designing private AI networking on Azure: isolation models, traffic flows, subnets, DNS, egress control, the AI gateway and a pre-go-live checklist.
Why AI Networking Is DifferentA classic web app has a predictable network shape: users come in through a WAF, the app talks to a database, done. An AI workload is noisier. There are more PaaS endpoints (models, search, storage, Cosmos DB, Key Vault), each with its own private DNS zone. Agents make outbound calls you didn't write yourself, to tools, MCP servers and sometimes the web. And some managed features run compute that must be injected into your network. Get the network design wrong and you'll find out in the worst way: an agent that works in dev and times out in production, because a DNS zone isn't linked or a firewall silently drops a token request.
The goalEvery AI call stays on private IPs, every name resolves to a private endpoint, and every outbound connection is one you explicitly allowed.
Step 1: Pick Your Isolation ModelThere are three realistic options on Azure today. Choose deliberately, because some of them can't be changed later.
My defaultFor enterprise landing zones: bring your own VNet. For a new team with no hub: managed VNet with allow only approved outbound. Whatever you choose, turn off public network access on every AI resource once private access works.
Step 2: Map the Traffic FlowsBefore drawing subnets, write down the five flows every AI workload has. Each one needs its own control.
1 · Inbound (users)
User
→
App Gateway / Front Door + WAF
→
Hub firewall
→
App (private endpoint)
2 · App → AI
App / agent
→
AI gateway (APIM)
→
Model private endpoint
3 · Agent → data
Agent
→
AI Search / Storage / Cosmos DB
→
via private endpoints
4 · Agent → tools (egress)
Agent subnet
→
Hub firewall (FQDN rules)
→
Approved APIs / MCP servers
5 · Identity & control plane
Every component
→
Microsoft Entra ID
→
(AzureActiveDirectory service tag)
Flow 4 is the one most designs forget. Agents call tools at runtime, and every tool is a potential exfiltration path. Flow 5 is the one that breaks deployments: block Entra ID and nothing can authenticate. 
Step 3: Plan the Subnets (Before Anyone Deploys)AI services are picky about subnets. The Foundry Agent Service subnet is the one that catches people out:
A starting layout for one AI workload spoke:
IP range gotchaThe agent subnet supports RFC 1918 ranges (
10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) and parts of 100.64.0.0/10. Some ranges are reserved and not supported, including 172.30.0.0/16 and 172.31.0.0/16. Check your IPAM plan before you hand out that /24.
Step 4: Get DNS Right (This Is Where It Usually Breaks)Private endpoints are useless if names still resolve to public IPs. For a Foundry-based workload you'll typically need these private DNS zones:
Step 5: Control EgressDefault-deny outbound is the whole point. In practice that means:
Don't forget the platformPrivate networking doesn't make every feature private. Some tools behave differently under network isolation (for example, certain file-upload flows in Code Interpreter), and web-grounding tools send queries outside your network by design. Check the limitations list before promising "fully private" to your security team.
Step 6: Put the AI Gateway on the Private PathIf you front models with Azure API Management, the tier decides your network options:
For an enterprise AI gateway, I use either internal-mode injection (Premium classic or Premium v2) or Standard v2 with a private endpoint inbound plus VNet integration outbound. Note that public network access on APIM can only be disabled after a private endpoint is configured. Step 7: Add a Perimeter for PaaS-to-PaaSA Network Security Perimeter (NSP) groups PaaS resources (Foundry, AI Search, Storage) into a logical boundary with inbound and outbound rules, and logs every access decision. Start in transition (learning) mode, review the logs, then switch to enforced. It only governs data-plane traffic, so it complements private endpoints rather than replacing them. The Pre-Go-Live Checklist
My TakeAI networking isn't harder than any other Private Link design, but it has more moving parts and less tolerance for shortcuts. Plan the subnets and DNS zones before the first deployment, treat agent egress as a security control rather than an afterthought, and test name resolution from inside the network on day one. That's 80% of the incidents I see, avoided. Earlier in this series: the AI Gateway and 10 decisions that make or break your AI platform.
References
Set up private networking for Foundry Agent Service Configure managed virtual network for Microsoft Foundry Configure network isolation for Microsoft Foundry Add Foundry to a network security perimeter API Management virtual network options Secure networking for Azure AI platform services (CAF) Networking recommendations for AI on Azure infrastructure (CAF) Baseline Foundry chat architecture in an Azure landing zone Integrate Container Apps with Azure Firewall
0 Comments
Leave a Reply. |
Author
Mohammad Al Rousan is a Microsoft Most Valuable Professional (MVP) in Azure, a cloud architect, and a recognized leader in enterprise AI and data platforms. With over a decade of hands-on experience, he specializes in designing and scaling secure, production-grade solutions across Azure AI, Databricks, and modern cloud-native architectures. 
Top 10 Microsoft Azure Blogs
Archives
September 2026
Categories
All
|
RSS Feed
