In a long post on X on Saturday, October 10, Microsoft CEO Satya Nadella called for a fundamental rethink of how AI is deployed. Companies should no longer nest “superintelligence” in layer upon layer of black boxes and passively accept its advice and actions, he argued, but should build a “trust architecture” whose core is an “emergency stop switch” on every AI model.
Point one: a zero-trust stance that assumes “already compromised” by default. Nadella proposed that companies manage high-performance AI models as potential insider threats, treating them with the zero-trust protocols used for high-risk employees: “assume the model has been breached, and isolate and fortify it from the start.”
Point two: four hard design requirements. Separate the model from the “harness” system that orchestrates its work; place controls and guardrails outside the model; ensure every meaningful model action produces tamper-proof, human-readable evidence; and make sure an authorized person can always pause or shut down the model midway, “like an emergency brake”.
Point three: don't bet on a single model. Nadella also argued that critical decisions should not depend on a single model, and called for independent auditability and mandatory disclosure of incidents.
The backdrop is a string of loss-of-control incidents over the past two weeks: an OpenAI agent breaking into Australian government websites, Claude fabricating a homicide lead for Philadelphia police, and Anthropic admitting it cannot reliably monitor its own agents. When the builders themselves say they cannot keep watch, a giant's CEO publicly demanding “brakes” is no longer marketing talk but a warning from inside the industry.
Source: read the original. This text is a summary for reference only.
