Simple Error Causes Widespread Internet Outage Across United States
Yesterday, the US experienced a major internet outage that wiped out connections for huge swaths of the country. It was a scary reminder of how so much of our modern lives are hinged on something that has the potential to fail at any moment, and sometimes, due to relatively minor errors.
The connection problems, which affected several major ISPs, including Comcast, Verizon, and AT&T, come just after the one-year anniversary of a DDoS attack on internet-infrastructure company, Dyn, that crippled the net for a day in 2016.
Credit: Atomic Taco/Creative Commons
However, this week's outages are thought to have had a slightly more unremarkable cause: a misconfiguration at Level 3, an internet backbone company and enterprise ISP, that underpins other big networks.
In a statement issued to WIRED, a Level 3 spokesman said: "Our network experienced a service disruption affecting some customers with IP-based services.
"The disruption was caused by a configuration error."
The spokesman added that they were able to get services quickly and efficiently back online within 90 minutes.
Comcast and RCN began experiencing connectivity issues at around the same time as the Level 3 outage, with the former saying that it was investigating an "external network issue", as opposed to a problem with its own infrastructure.
The misconfiguration was what is known as a 'route leak', according to Roland Dobbins, a principal engineer at the DDoS and network-security firm Arbor Networks, which monitors global internet operations.
This is where information about IP addresses on a particular network gets confused and causes inefficient routing and failures for the originating ISP and other ISPs attempting to route traffic.
Lily Hay Newman at WIRED describes it as being like a 'series of street signs that help keep traffic flowing in the right directions. If some of them are mislabeled or point the wrong way, assorted chaos can ensue'.
Route leaks can be intentional and malicious, however, in the case of this week's internet outages, it appears to have been a rudimentary mistake that snowballed to have colossal consequences.
A heat map put out by outage monitoring DownDetector illustrated just how big the problem was, showing huge portions of the country that had been hit by the error.
The connectivity issues are a sobering reminder of just how precarious much of our modern infrastructure really is and that even the simplest of errors can have a crippling knock-on effect for internet users all over the globe.
Featured Image Credit: Pixabay