Microsoft CEO Nadella: Agentic AI needs healthy dose of deterministic controls

Microsoft CEO Satya Nadella said agentic AI systems need to separate architecture and controls from frontier models and leverage deterministic controls to keep AI agents in check.

In a post on X, where tech CEOs apparently play philosophers most weekends, Nadella said, "we can't outsource responsibility for what intelligence does on our behalf."

He said:

"We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions. We must build contained systems whose behavior we can observe, limits we can test, and actions we can always contain."

In other words, the authority and intelligence need to be separated.

• Nvidia launches Open Agent Safety Platform to secure AI agents
• Personal AI assistants to proliferate and so will the governance headaches
• Agentic AI deployments: What you should, and shouldn't do
• Why every AI agent should be treated like an insider threat

Nadella's message isn't unique but reinforces what Nvidia recently outlined with its Open Agent Safety Platform. Nadella also said it makes sense that AI agents should be treated like insider threats, which is something we've noted in recent weeks too.

"We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient.

Treating frontier closed and open weight models like insider risks is a way to build such a system. Not because they are necessarily malicious, but because any sufficiently capable actor with access to important systems can make mistakes or be compromised, and the architecture of containment and control must account for that."

Nadella said model diversity, observability at every layer, continuous testing, independent controls and audits and containment are required for agentic AI systems.

Mustafa Suleyman, CEO of Microsoft AI, noted that controlling AI systems is an engineering and governance challenge.

A few things to note here.

• The enterprise industry appears to be firmly aligned against OpenAI and Anthropic when it comes to regulation and controlling models. Nvidia and Microsoft both own shares of both frontier model labs and are indicating both need to grow up.
• Governance is the word of late 2026 and 2027.
• Aside from deterministic controls, this governance could easily be solved by a barrage of lawsuits for lax security. In the end, a human will always be the fall guy when a mode runs amok.