Powering the technical veracity of AI at Microsoft with a Center of Excellence

When we launched our AI Center of Excellence (CoE) in 2023, we had a straightforward goal: Help our organization experiment with AI, learn quickly, and do it responsibly.

Our teams across Microsoft Digital—the company’s internal IT organization—leaned in. We built tools, workflows, and AI enabled solutions at speed. Momentum followed, along with real enthusiasm and growth.

“We did a lot of good work building community and excitement. But at some point, we needed to evolve and put more structure around what we’d built.”

Qingsu Wu, principal group product manager, Microsoft Digital

But increasing scale required us to evolve our approach.

As adoption accelerated, we began to see duplication, uneven governance, and growing gaps between strategy and delivery. What helped us move fast early on wasn’t enough to sustain impact over time.

“We did a lot of good work building community and excitement,” says Qingsu Wu, a principal group product manager who leads the AI CoE at Microsoft Digital. “But at some point, we needed to evolve and put more structure around what we’d built.”

AI agents and solutions began appearing across Microsoft Digital. Different teams solved similar problems. Standards were interpreted differently. Reporting was inconsistent, and in many cases manual.

The question was no longer, “How do we help teams try AI?” It became, “How do we turn AI into consistent, measurable outcomes at scale?”

Answering that question required a change in how our CoE operated.

Rather than acting primarily as an advisory group, the AI CoE evolved into an execution‑focused function. Its role expanded from guidance to coordination, helping set priorities, define guardrails, and connect AI work directly to business outcomes.

The goal wasn’t to slow AI innovation down, but to help it move in the correct direction with more agility and better scalability.

Evaluating AI for Microsoft

The AI CoE connects AI strategy to execution across Microsoft Digital. It operates as a cross‑functional coordination layer that sets direction and creates shared accountability for how AI work gets done.

“We can see patterns that a single team can’t. We’re translating AI CoE strategy and enterprise priorities into clear execution plans that work in each organization’s context. That helps us align priorities and make sure the biggest bets are actually landing.”

Ria Khetan, senior program manager, Microsoft Digital

The CoE brings our leaders and practitioners together from AI, data, responsible AI, and operations to answer questions collectively. We use that cross‑disciplinary view to operate above individual projects without losing touch with day‑to‑day reality.

The CoE looks across the organization and answers questions individual teams can’t answer on their own.

• What AI initiatives are already in flight?
• Which ones matter most to the business?
• Where are teams duplicating effort?
• Where do we need clearer standards or stronger governance?

“We can see patterns that a single team can’t,” says Ria Khetan, a senior program manager in Microsoft Digital who helps lead program management for the AI CoE. “We’re translating AI CoE strategy and enterprise priorities into clear execution plans that work in each organization’s context. That helps us align priorities and make sure the biggest bets are actually landing.”

We’ve designed the AI CoE to act as the connective tissue between leadership intent and execution on the ground. It helps ensure that AI work across Microsoft Digital moves forward with purpose, consistency, and measurable impact.

Building transformation on core pillars

The AI CoE establishes a common structure that helps our teams work toward the same outcomes, even when they are building different solutions.

“We use the CoE to bring consistency to how AI work gets done. It gives us a way to step back and ask whether we’re solving the right problems and whether we’re set up to scale.”

Don Campbell, principal group technical program manager, Microsoft Digital

The operating model is intentionally simple.

AI initiatives are reviewed against shared pillars that help teams think beyond individual projects. These lenses ensure the work aligns to business priorities, can scale safely, has a clear delivery path, and supports responsible adoption.

“We use the CoE to bring consistency to how AI work gets done,” says Don Campbell, a principal group technical program manager who leads AI strategy here in Microsoft Digital. “It gives us a way to step back and ask whether we’re solving the right problems and whether we’re set up to scale.”

Our CoE uses these four pillars to guide our work:

• Strategy. We work with product and feature teams to determine what we want to achieve with AI. They define business goals and prioritize the most important implementations and investments.
• Architecture. We enable infrastructure, data, services, security, privacy, scalability, accessibility, and interoperability for all our AI use cases.
• Roadmap. We build and manage implementation plans for all our AI projects, including tools, technologies, responsibilities, targets, and performance measurement.
• Culture. We foster collaboration, innovation, education, and responsible AI among our stakeholders.

These pillars are the common language that helps us connect strategy to execution and make decisions across all teams and scenarios at Microsoft Digital.

Strategy

Our CoE strategy team’s role is to step back and create clarity.

Our strategy is driven from the organization’s top level, and executive sponsorship is crucial to executing our implementation well. When our transformation mandate comes from the organization’s leader, it resonates in every corner of the organization, every piece of work, and every task. We also encourage and welcome ideas from every level of the organization, empowering individuals to contribute their AI insights.

We maintain a centralized view of AI initiatives across Microsoft Digital, including agents, workflows, and AI‑enabled solutions. That visibility allows our CoE team to identify duplication, surface opportunities to scale successful ideas, and align investments to enterprise priorities. This creates a shared intake and prioritization model.

One of our CoE strategy team’s most significant responsibilities is prioritizing the idea pipeline for AI solutions. All employees can feed ideas into the pipeline through a form that records important details. The strategy team then evaluates each idea, analyzing two primary metrics:

• Business value. How important is the solution to our business? Potential cost reduction, market opportunity, and user impact all factor into business value. As our business value increases, so does the idea’s position in the pipeline priority queue.
• Implementation effort. We focus on clearly defining the problem statement—what the problem is, why it matters, who the customer is, the baseline metrics, and the plan to attribute value pre‑production. This ensures we prioritize AI for the most critical business problems and can measure impact before and after deployment.

By anchoring AI work in business outcomes from the start, the strategy pillar helps ensure the organization’s energy is spent on the work that matters most.

Architecture

Our architecture pillar defines how we help teams scale AI solutions without creating security gaps, compliance issues, or technical debt they’ll have to unwind later.

“The CoE introduces a framework to enable design reviews in the early development phase. We help make sure teams are choosing the right platforms and thinking about security and compliance from the beginning.”

Qingsu Wu, principal group product manager, Microsoft Digital

Before solutions move into broader use, our architecture team helps think through data readiness, platform alignment, and governance requirements. The goal isn’t to prescribe a single architecture, but to make sure foundational decisions won’t limit scale or create risk down the line. Many times, this means doing things before development, while other times it means making improvements after the initial development is done and the product or scenario is launched and being used. We also track our efforts with measurable metrics like usage.

One common pitfall is that teams may gravitate toward the most flexible platforms with full control, without fully understanding the associated security and compliance implications. To address this, we publish clear guidance to help teams choose the right platform—one that strikes the appropriate balance between flexibility and the security and compliance effort required.

Our architecture pillar helps prevent that by reinforcing a set of common expectations. Teams still build locally and move fast, but they do so within a framework that supports reuse, interoperability, and responsible operation built on enabling teams and employees to experiment with guardrails that keep our production systems safe.

“The CoE introduces a framework to enable design reviews in the early development phase,” Wu says. “We help make sure teams are choosing the right platforms and thinking about security and compliance from the beginning.”

Teams are encouraged to build on recommended platforms and services that support enterprise‑grade security, observability, and lifecycle management. This helps ensure solutions can be monitored, governed, and supported over time.

Security and compliance are never treated as downstream checkpoints. Architectural guidance reinforces the need to design with identity, access controls, auditability, and responsible AI principles from the start.

When solutions prove valuable, we look for opportunities to reuse architectural patterns, components, or services rather than rebuilding them in isolation. This reduces duplication and accelerates future work.

Roadmap

Our CoE roadmap team examines our employee experience in the context of our AI solutions and governs how we achieve the optimal experience in and throughout AI projects. It focuses on how our employees will interact with AI. Getting the roadmap right ensures user experiences are cohesive and align with our broader employee experience goals.

We’ve recognized AI’s potential to impact how our employees get their work done.

Their experiences and satisfaction levels with AI services and tools are critical. Our roadmap pillar is designed to encourage experiences across all these services and tools that are complementary and cohesive.

We’re focusing on the open nature of AI interaction.

“We’re surfacing AI capabilities and information when the user needs them, according to their context,” Campbell says. “It makes the user experience and user interface for an AI service less important than how the service allows other applications or user interfaces to interact with it and harness its power.”

A key part of this approach is disciplined experimentation.

Rather than treating every idea as a long‑term commitment, the roadmap pillar helps teams validate value early. Our teams know when they’re in an experimental phase and when they’re expected to operationalize. This gives our leaders a more consistent view of progress and risk. The net result is that dependencies between teams surface earlier, when they’re easier to resolve.

Culture

Our culture pillar ensures that AI adoption across Microsoft Digital is intentional, responsible, and sustainable.

Culture underpins everything we do in the AI space. Ensuring our employees can increase their AI skillsets and access guidance for using AI responsibly are critical to AI at Microsoft.

“We’re driving a shift from ad‑hoc AI usage to intentional, outcome‑driven adoption,” Khetan says. “That requires clarity, education, and shared expectations.”

In practice, that means the culture pillar defines how our teams are expected to adopt AI and integrate it into their work, not just what tools they can use.

Our culture team works with AI champions across the organization to translate enterprise AI priorities into local execution. Those champions act as two‑way conduits, bringing real‑world feedback and blockers back to the CoE and carrying guidance, standards, and learnings back to their teams.

Without this structure, AI adoption tends to fragment as teams experiment in isolation.

Our culture team has published training, recommended practices, and our shared learnings on next-generation AI capabilities. We work with individual business groups at Microsoft to determine the needs of all the disciplines across the organization. That work extends to groups as diverse as engineering, facilities and real estate, human resources, legal, sales, and marketing, among others. 

Responsible AI is embedded throughout that work.

The CoE reinforces responsible AI practices as part of everyday decision‑making—during design, experimentation, and scale. Teams are expected to understand not just what they’re building, but the implications of how they build it.

In the AI CoE, culture isn’t abstract. It shows up in how teams propose ideas, how they design solutions and how they measure success.

Fostering agent innovation

The true value of the AI CoE is evident when strategy, architecture, roadmap, and culture come together around real work.

A clear example of that is how we addressed the rapid growth of AI agents across the organization.

“That’s the core problem we’re trying to solve. In the past, admins had to go to multiple portals just to understand how many agents exist, and they all give different answers.”

Garima Tiwari, principal product manager, Microsoft Digital

Our teams were building agents in different platforms, for different scenarios, and at very different levels of maturity. That flexibility accelerated innovation, but it also made it difficult to answer basic questions.

• How many agents exist today?
• Which ones are in production?
• Which ones touch sensitive data?

The strategy lens helped clarify what mattered most. Our goal wasn’t to inventory every experiment. It was to gain visibility into agents that were active, scaling, or depended on by others, and to ensure those agents aligned to business priorities and Responsible AI expectations.

Architecture quickly followed.

As the CoE looked at how agents were built, we quickly discovered that information about agents was fragmented across tools. Different platforms showed different numbers. Ownership wasn’t always clear. And governance signals were hard to reconcile.

“That’s the core problem we’re trying to solve,” says Garima Tiwari, a principal product manager in Microsoft Digital leading our internal strategy and adoption of Agent 365. “In the past, admins had to go to multiple portals just to understand how many agents exist, and they all give different answers.”

This is where Agent 365—which we use to govern agents here at Microsoft—became a critical enabler.

Agent 365 brings together signals from multiple agent‑building platforms into a single, consolidated view. That visibility allows the CoE and administrators to understand agent inventory, ownership, lifecycle state, and governance posture in one place.

“Agent 365 is really about accurate inventory and observability,” Garima says. “It provides one number we can trust and a way to see how agents are behaving, who they’re interacting with, and whether they’re compliant.”

That architectural clarity changed how decisions were made.

Instead of guessing what was safe to scale, the CoE could see which agents were production‑ready, which needed remediation, and which should remain in experimentation. Security, privacy, and compliance considerations moved to earlier in the lifecycle.

“We can’t scale what we don’t understand,” Wu says. “Agent 365 helps us see what’s actually running so we’re not scaling something blindly.”

The roadmap lens then brought structure to execution.

“What changed was the mindset. Teams started thinking about manageability, security, and scale much earlier, not after an agent was already deployed.”

Don Campbell, principal group technical program manager, Microsoft Digital

Rather than standardizing everything at once, the CoE helped teams sequence work. Some agents stayed in pilot. Others moved toward broader rollout, informed by architectural and governance signals surfaced through Agent 365.

Culture and enablement ran alongside that work.

Teams began factoring operational readiness into design decisions instead of treating governance as a final checkpoint. Agent 365 isn’t positioned as a control tool at the end of the process, but as part of building agents the right way from the start.

“What changed was the mindset,” Campbell says. “Teams started thinking about manageability, security, and scale much earlier, not after an agent was already deployed.”

The outcome wasn’t a single standardized solution.

It was a repeatable approach within a shared CoE framework, supported by platforms like Agent 365, that made scaling AI more visible, more manageable, and more intentional.

That’s what the AI CoE enables at Microsoft Digital.