Concise Cyber

Subscribe below for free to get these delivered straight to your inbox

Advertisements
Fastly Outage Explained: How a Single Bug Triggered a Global Internet Shutdown
Advertisements

On June 8th, 2021, a significant portion of the internet went dark as popular websites and services suddenly became inaccessible. The culprit was not a malicious cyberattack but a single, latent software bug within the infrastructure of content delivery network (CDN) provider, Fastly. The outage impacted major platforms including Reddit, Spotify, Twitch, PayPal, and news outlets like CNN and The New York Times.

The Trigger: A Dormant Bug Awakened

The root cause of the widespread outage was a software bug introduced in a Fastly deployment on May 12th. The flaw remained dormant and undetected for weeks. According to a detailed post-mortem by Fastly’s SVP of engineering and infrastructure, Nick Rockwell, the bug was triggered on June 8th when a single, unnamed customer made a valid configuration change. This specific change exposed the bug, which led to 85 percent of Fastly’s network returning errors and effectively taking down client websites.

Timeline of Detection and Recovery

Fastly’s engineering teams responded swiftly to the cascading failure. The company detected the disruption within one minute of its onset at 9:47 UTC. Engineers identified the root cause of the problem at 10:27 UTC and began rolling out a fix. By 10:36 UTC, the patch was deployed, and by 11:00 UTC, 95 percent of the network was restored to normal operation. Fastly reported a full recovery by 12:35 UTC. “This outage was broad and severe, and we’re truly sorry for the impact to our customers and everyone who relies on them,” Rockwell stated.

Source: https://biztoc.com/x/b73515925b891f2f