Before AI Touches Your Infrastructure, Make Sure You Can Roll Back

Stop and Think Before AI

AI is becoming the new default way engineering teams work. Tools like Claude Code, Codex, Cursor, and other AI agents can help teams write code, review changes, generate Terraform, and move faster across cloud and infrastructure workflows.

But AI comes with a toll: risk.

AI agents can misunderstand context, make confident assumptions, or hallucinate details that look correct but are wrong. In some cases, the mistake has no real impact. In other cases, it can touch production infrastructure and create disruption.

Imagine a cloud engineer using Claude with AWS access through MCP to speed up infrastructure work. The agent uses the wrong credentials, operates in the wrong AWS account, and deletes a CloudFront distribution in production.

At that moment, the question is not “Is AI useful?”
The question is: “How quickly can we recover?”

Using AI is becoming the de facto way to work. Organizations should not ignore that. But before giving AI agents access to cloud environments, CI/CD pipelines, or Infrastructure as Code workflows, they need the right safety net.

They need visibility into what changed, controls around what AI can do, and most importantly, a way to restore the last working state when AI makes a mistake..

The Moment Before Moving Fast

Northstar Payments had a clear goal: speed up its Infrastructure as Code development.

The company was growing, and its cloud environment was growing with it. New services needed to be deployed faster, and AI looked like an obvious answer.

The engineering team had already tested AI coding tools for application development, and the results were promising. Developers were moving faster. So why not use AI to help with cloud infrastructure too?

On paper, it made sense. An AI agent could help write IaC modules, review policy changes, suggest fixes, and reduce the manual effort behind infrastructure updates.

But Northstar did not rush.

The leadership team had heard enough stories to make them cautious. AI agents given more access than intended and Engineers approving actions they did not fully read led several companies to configuration changes that looked harmless but created critical business  exposure. 

While none of these stories meant AI should be avoided, they made one thing clear: using AI for infrastructure is not the same as using AI to draft a function or clean up documentation.

Infrastructure carries business risk. A small change can affect availability, security, cost, or compliance. When AI enters that workflow, risk management must accompany it.

So Northstar paused.

How do we move faster without giving up control?

The Case That Almost Stoped One Dollar 

One Dollar Retail had already learned this lesson the hard way.

A platform engineer used an AI agent to help clean up a Terraform module that had become difficult to maintain. The request sounded routine: simplify the code, remove old variables, and align the module with the company’s current cloud standards.

The AI produced a large change. It looked organized. The explanation was confident. The engineer reviewed the summary, skimmed the details, and approved the update.

What nobody noticed at first was that the change modified how a shared cloud role was created. That role was used by more than one environment. When the pipeline ran, the change did not only affected one application. It also disrupted access used by a production workloads and cost the company money and time.

You might ask yourself Why Someone Would Skim the Change

In One Dollar’s case, the engineer was not careless. He was overloaded.

We all been there, he had several internal teams waiting for infrastructure updates, a security ticket that needed attention, and a backlog of Terraform cleanup work that had been postponed for months. AI looked like a way to finally move faster.

That is part of the challenge.

If the engineer had to read every line of the AI-generated change with the same depth as writing it from scratch, much of the productivity gain would disappear. At some point, reviewing the AI output can start to feel almost like doing the work manually.

So he did what many experienced people do under pressure: he reviewed the summary, checked the parts that looked important, scanned the diff, and moved on.

That behaviour is human.

From Blocking AI to Controlling Risk

Northstar could have taken the simple path and completely blocked AI from infrastructure work, but Northstar chose a different approach:

Instead of asking, “How do we prevent AI from ever making a mistake?” they asked a better question: If AI makes a bad change, how quickly can we detect it, understand it, and restore what was affected?

That shift changed the discussion.

The goal was no longer to trust AI blindly or reject it completely. The goal was to build a safety layer around AI-assisted infrastructure work. If an agent suggested the wrong change, used the wrong permission, or helped create a risky configuration, the organization needed a way to recover.

This is where AI adoption becomes a risk management exercise.

Before giving AI access to infrastructure workflows, organizations should ensure they have the basics in place: visibility into changes, and a reliable way to restore infrastructure as needed.

What Good Configuration Backup and DR Should Provide

For AI-assisted infrastructure work, backup and disaster recovery cannot be treated as an afterthought.

The organization needs one simple capability: the last working configuration should always be available, and ready to restore.

That means more than storing old Terraform files or keeping a backup somewhere. Teams need a time machine view of their infrastructure configuration. They need to see how the environment looked before a change, what changed over time, and which version represented the last known-good state.

When an AI-generated change causes a problem, teams should not be forced to investigate from zero while production pressure is rising. They should be able to quickly understand:

• What changed
• What the last working state was

This is the heart of configuration backup and disaster recovery.

The goal is not only to keep a copy of the code. The goal is to preserve operational confidence. Then, AI becomes less frightening.

Why Northstar Chose ControlMonkey

Northstar did not want to block AI from infrastructure work forever. The company wanted the speed, but only after it had a way to control the risk.

The team realized that approvals and code review were important, but not enough. AI could still generate a risky change. A busy engineer could still miss something that looked safe but created unexpected impact.

So Northstar looked for a solution that could answer a different question:

If something goes wrong, can we see exactly what changed and restore the last working configuration quickly?

That is where ControlMonkey became relevant.

ControlMonkey gives teams a resilience layer for cloud and SaaS configuration. It continuously captures the environment’s state, tracks changes over time, and creates recoverable versions of the infrastructure configuration. In practical terms, it gives the organization a “time machine” for cloud configuration, not just a backup folder.

For Northstar, this mattered because the risk was not only in the Terraform code. The real risk was in the live environment: IAM, networking, dependencies, unmanaged resources, drift, and changes that happened outside the normal IaC workflow.

ControlMonkey helped them in three important ways.

First, it gave visibility. The team could see what existed in the cloud, what was managed by IaC, and where there were gaps. That reduced the chance that an AI-assisted change would affect something the team did not even know was there.

Second, it gave history. Instead of asking engineers to remember what the environment looked

Third and must important :  it gave them last known good state to go back to.