Design an Azure landing zone for enterprise AI

An AI landing zone is a responsibility model

An Azure landing zone provides the governed cloud foundation in which a workload runs. The AI workload still needs its own architecture, reliability targets, threat model, and operating procedures. Begin by separating the shared platform services from the resources the AI workload team must own and change.

• Platform landing zone: shared connectivity, identity foundations, policy, and common operations capabilities
• Application landing zone: one or more subscriptions in which the AI workload runs
• Workload architecture: application, Foundry resources, models, retrieval, tools, data stores, and workload monitoring

Define ownership before selecting services

Write a responsibility matrix before drawing the final topology. Microsoft guidance for the Foundry landing-zone baseline treats the workload as the owner of its Foundry resource while shared platform teams operate common resources. Centralizing every AI resource can appear efficient, but it can blur cost allocation, change ownership, reliability dependencies, and incident responsibility.

• Who approves model and region use?
• Who owns the Foundry resource, projects, deployments, quotas, and upgrades?
• Who operates private DNS, routing, firewall, and cross-premises connectivity?
• Who owns retrieval indexes, source-data permissions, retention, and deletion?
• Who responds to safety, quality, security, and availability incidents?

Organize subscriptions around isolation

Use management groups for broad workload archetypes and inherited governance, not as a substitute for application design. For production-bound workloads, separate subscriptions can isolate environments, access, quotas, policy scope, and accidental deployment impact. Keep development, test, and production under the same workload archetype so code meets consistent guardrails throughout promotion.

• Use sandbox subscriptions for exploration that is not promoted directly to production
• Prefer a separate subscription for each application environment when isolation is required
• Avoid a management group for every workload or lifecycle stage
• Use subscription vending or another repeatable provisioning process
• Define budget ownership and cost tags when the subscription is created

Make identity the first security boundary

Use Microsoft Entra ID, Azure role-based access control, and managed identities wherever supported. Human administrators, deployment pipelines, serving applications, evaluation jobs, and data-ingestion processes have different responsibilities and should not share one broad identity.

• Assign roles to groups rather than directly to individual users
• Use just-enough scope and just-in-time privileged access for administration
• Create separate groups, service principals, and managed identities for each environment
• Keep deployment identity permissions separate from runtime data access
• Record and periodically review privileged and cross-subscription assignments

Design connectivity and DNS as one system

A private endpoint alone does not create a functioning private architecture. Map every traffic path, its direction, identity, name resolution, inspection point, and failure behavior. The application landing zone can connect to shared hub services while keeping workload-specific private endpoints and subnets under workload ownership.

• Inbound user or API path
• Application-to-model and application-to-agent path
• Retrieval and data-store paths
• Tool and external API egress
• Build, package, and deployment paths
• Private DNS resolution across hub, spokes, and connected networks

Establish data and model boundaries

Document which data classifications the workload may process, where prompts and outputs can be stored, and which services or models may receive them. Model access is only one boundary: ingestion, indexes, logs, traces, evaluations, tool calls, caches, and human review queues can all contain sensitive information.

• Approved regions, model providers, and deployment types
• Source ownership, permission propagation, freshness, and deletion
• Prompt, response, trace, and evaluation retention
• Encryption and customer-managed key requirements
• Content filtering, abuse monitoring, and human escalation
• Rules for external tools and cross-boundary data transfer

Use policy as a tested guardrail

Apply policy at the highest appropriate scope and keep the baseline consistent across lifecycle environments. A guardrail discovered only during production deployment creates rework. Test infrastructure modules against inherited policies early and define a visible exception process for requirements that the standard platform does not support.

• Allowed regions and resource types
• Required private connectivity and public-network restrictions
• Diagnostic settings and approved logging destinations
• Required tags, budgets, and ownership metadata
• Security services and configuration auditing
• Time-bound exception owner, rationale, risk, and expiry

Build an operations baseline and workload SLOs

The platform provides common inventory, compliance, monitoring, and recovery capabilities; the workload team adds AI-specific signals. Monitor system outcomes instead of only resource health so an available endpoint with failing retrieval or unsafe tool behavior is not reported as healthy.

• Availability, latency, throttling, dependency, and quota signals
• Model, prompt, retrieval, tool, and index versions
• Task success, groundedness, safety, and approval compliance
• Token, search, storage, networking, and evaluation cost
• Recovery objectives, backup scope, rollback, and model-deployment fallback
• Runbooks with platform and workload escalation paths

Create a paved road, not a central bottleneck

Package the agreed decisions as versioned infrastructure modules, policy tests, reference pipelines, monitoring defaults, and architecture decision templates. Workload teams should be able to deploy within guardrails without opening a platform ticket for every change, while platform teams retain visibility and control over shared-risk boundaries.

• Subscription and environment request
• Approved infrastructure modules
• Identity and network bootstrap
• Policy-compliant CI/CD pipeline
• Standard dashboards, alerts, and cost controls
• Documented exception and decommissioning workflows

Review the design with failure scenarios

Before production, walk through failures that cross team boundaries: private DNS breaks, a model deployment reaches quota, a retrieval index exposes unauthorized content, an agent tool becomes unavailable, a policy update blocks deployment, or a region is unavailable. Record which team detects the problem, who owns mitigation, and how the workload degrades safely.