Block operates one of the most advanced cloud platforms in the fintech space. With products like Square and Cash App driving billions in transactions, uptime, scale, and resilience aren’t just technical goals—they’re business imperatives.
In late 2023, as part of Block’s commitment to operational excellence, leadership at one business unit initiated a comprehensive review of their systems, services, and infrastructure to identify opportunities for enhanced resilience.
The review surfaced a critical need: while data backups were in place, the infrastructure configuration itself—networking, security, IAM, routing—had no guaranteed path to recovery.
The team wanted a new solution to answer questions like:
- What exactly is covered by Terraform—and what isn’t?
- What if an IAM policy or security group is deleted?
- Could we rebuild our infra the way it was, in the right order, under pressure?
That moment became a turning point. Therefor Block’s Platform Engineering team began looking for a solution that could give them total confidence in their infrastructure—not just when things were working, but when they weren’t.
Block challenge: Complex infrastructure, unknown coverage.
At Block’s scale, infrastructure evolves rapidly. Teams innovate quickly across hundreds of AWS accounts, using a combination of Terraform, automation scripts, homegrown tooling, and various deployment methods. As the platform matured, the team recognized an opportunity to enhance visibility and standardization.
Like many fast-growing organizations, some resources had been deployed through various methods, and the team saw an opportunity to improve consistency in tagging and tooling. This presented a chance to enhance coverage and traceability across the infrastructure.
“As our platform evolved over the years, we recognized we had an opportunity to improve our infrastructure visibility,” said Ben Apprederisse, Platform Technical Lead at Block. “We wanted complete confidence in our disaster recovery capabilities, which meant ensuring every resource was properly tracked and recoverable.”
Block’s vision extended beyond traditional backups. The team wanted to establish a new standard with complete visibility, consistent coverage, and reliable configuration recovery capabilities—setting the foundation for even greater operational resilience.
The solution: A new standard for infrastructure recovery
The team explored several tools—but most were too brittle, too incomplete, or too slow to deploy. What they needed was a platform that could show them exactly what was running, compare it to their Terraform code, and give them reliable, validated configuration snapshots they could use to recover fast. They chose ControlMonkey.
ControlMonkey gave Block the ability to:
- Continuously scan infrastructure across all AWS accounts
- Generate clean, validated Terraform code for any existing resource
- Automatically snapshot infrastructure configuration daily—creating a recovery point for every asset
- Provide dashboards showing IaC coverage, and unmanaged infrastructure
What stood out wasn’t just the technology—it was how quickly they could get it working. One engineer was able to roll out full coverage across key accounts in exactly 2 weeks.
“We needed something fast, reliable, and easy to run,” said Ben. “ControlMonkey gave us all of that—and more.”
Results: Block Gained Cloud resilience, visibility, and confidence
With ControlMonkey, Block transformed the way it thinks about infrastructure resilience. What began as a Cloud DR project quickly became a broader initiative to unify visibility, enforce standards, eliminate blind spots, and slowly ramp iac coverage.
Already, the platform has delivered meaningful impact:
- Infrastructure snapshots reduced recovery time
Recovery of resources that once took hours now only requires looking at the Git diff between snapshots. - Coverage insights revealed hidden risk
Teams learned just how much infrastructure wasn’t in code—and began fixing it. - Confidence in change
Developers can now see what’s changing, when—without relying on tribal knowledge.
“It’s not just about backing things up. It’s about knowing what’s covered, where the gaps are, and how to fix them. That’s a different kind of confidence,” said Ben.
What’s next?
Block is now expanding its use of ControlMonkey across the full company—including to GCP accounts—and working to eliminate unmanaged infrastructure entirely. The goal isn’t just recovery—it’s maturity.
Next steps include:
- Scaling Terraform import to bring legacy resources under governance
- Introducing proactive drift detection and alerting
- Simulating targeted DR events to test real-time recovery capabilities
Block isn’t just building resilience—it’s raising the bar for how cloud infrastructure is managed at scale. By focusing on clarity, coverage, and control, the team is creating an environment where developers move faster—and the business moves forward with confidence. And ControlMonkey is powering that shift.
“We needed something fast, reliable, and easy to run,” said Ben. “ControlMonkey gave us all of that—and more.”
Author
