Anthropic engineer’s warning about Claude Code explains why tech outages could get much worse

There have been so many prominent tech services experiencing outages lately that the problems have gone mainstream in a new way we haven’t seen before. 

Besides the headlines and podcasts (some of which are asking if this is actually a new thing or if we’re just noticing it more), the tech outages have kept Vice President JD Vance from attending New York events, led to the resignation of the White House’s cybersecurity chief, made “Saturday Night Live,” and been the subject of tons of memes and TikTok comedy where people joke that “LinkedIn and Tinder must be down, because I’m not getting any responses!”

As we wrote recently, the problems are far more than just cybersecurity, though AI means hackers can discover and exploit bugs like never before. Infrastructure is decrepit. Maintenance is deferred. Exhausted, too-small teams of IT workers aren’t up to this level of chaos.

But there’s another explanation that’s perhaps underappreciated: one big downside of Claude Code, the marquee product that has given Anthropic (ANT) its trillion-dollar valuation.

The problem is best explained by someone who would know best: a prominent Anthropic engineer (and former Google engineer) who works on Claude Code as a member of the technical staff, Addy Osmani, whose treatise on “comprehension debt” bounced around the software engineering community in March — and is even more relevant today.

What Osmani highlights isn’t really a weakness in Claude Code, but in how companies and engineering teams use it. The gist: AI can write code faster than we can review it. The speed is a great benefit, but it’s also a curse, as myriad companies pivot from writing code to overseeing teams of AI agents that complain less and write more.

As Microsoft CEO Satya Nadella said this week, the company’s engineers “build a software factory that builds the product. They don’t build the product.”

Dario Amodei is seen on the display at the Nasdaq MarketSite as shares of Anthropic begin trading after their IPO in Times Square, New York City, United States, on November 19, 2026. The IPO for the artificial intelligence company that created the AI model Claude made its Nasdaq debut under the ticker ANT, opening at $111.50 per share after pricing its $81.3 billion IPO at $83.00 per share. (Photo by Stephanie Keith/NurPhoto via Getty Images)
Dario Amodei is seen on the display at the Nasdaq MarketSite as shares of Anthropic begin trading after their IPO in Times Square, New York City, United States, on Nov. 19, 2026. (Stephanie Keith/NurPhoto via Getty Images) · NurPhoto via Getty Images

Traditionally, the laborious process of checking other people’s code is how software engineers internalize why systems are designed the way they are, and why decisions from earlier in a project might be meaningful today.

But reviewing the entirety of AI code isn’t feasible given how fast we can produce it now. It’s too unwieldy, and this represents a massive bottleneck for progress. Tests, AI review, and spending less time reviewing can be effective. But they’ve removed layers of verification, and ultimately comprehension, which is disappearing, Osmani explains.

All this means that with code piling up faster than humans can review it, companies can lose control of their systems in a way far more prosaic and banal than some sort of sentient AI or Skynet doomsday scenario.

With human code proofreaders at their limits, when things inevitably break, that missing understanding matters.

Coders — or even AI — may not realize that a few lines of code represent an important piece of figurative duct tape, a load-bearing wall, or a mechanism that triggers other contingencies. Every patch or attempted fix can also break more things.

That brings us back to the wave of cyber outages.

Aside from a few high-profile lawsuits and public scandals, the tech giants whose software “ate the world” and that now commandeer our attention spans have largely escaped meaningful accountability. Their software failures are embarrassing. And their design choices have led to allegations of addiction, anti-social behavior, and economic exploitation. But they have, generally speaking, escaped rigorous regulation.

AI changes the calculus. The era of honor system self-policing may be coming to an end. “When AI-generated code is running in healthcare systems, financial infrastructure, and government services,” Osmani wrote, the excuse that an AI wrote bad code and we didn’t fully review it “will not hold up in a post-incident report when lives or significant assets are at stake.”

Cyber outages are often a nuisance and, mercifully, haven’t led to a mass-casualty event or prolonged economic crisis. But in a world where AI can generate code faster than anyone can keep up, it’s not hard to imagine that changing.

Source link

Tagged:

Leave a Reply

Your email address will not be published. Required fields are marked *