Maintenance Error, Faulty Configuration change caused Facebook Data Centres to Malfunction

The outages which users of Facebook, Whatsapp and Instagram faced on Monday night could be due to DNS Disruption, ‘Hacktivism’ and BGS Updates, cyber security experts said.

“Billions of users have been impacted by the services being entirely offline today,” tracker Downdetector wrote on its website. Cyber experts are of the view that a combination of DNS Disruption & BGS Update caused this 7 hour long outage.

DNS Disruption & BGS Update Issues

DNS mishaps are common enough, and when in doubt, they’re the reason why a given site has gone down. They can happen for all kinds of technical reasons, often related to configuration issues, and can be relatively straightforward to resolve. In this case, the issue seems to be more complex.

“Facebook’s outage appears to be caused by DNS; however that’s a just symptom of the problem,” said Troy Mursch, chief research officer of cyber threat intelligence company Bad Packets in a report by WIRED.

The fundamental issue, Mursch said is that Facebook has withdrawn the Border Gateway Protocol (BGP) route that contains the IP addresses of its DNS nameservers. If DNS is the internet’s phone book, BGP is its navigation system; it decides what route data takes as it travels the information superhighway.

So, this raises the question as to why was the BGP withdrawn? Once Facebook came back online, it sent a statement that still lacked any technical detail. “To everyone who was affected by the outages on our platforms today: we’re sorry,” the company said. “We know billions of people and businesses around the world depend on our products and services to stay connected. We appreciate your patience as we come back online.”

There is also a political angle that can never be confirmed but is gaining rapid attention. “This kind of visible attack, where services are unavailable, tend to fall more in the social and political commentary segment of hacking or hactivisim,” according to Jake Dewoskin, a cybersecurity expert with Code 42, who was quoted in FOX9.

The social media giant on its part said on Monday that a global outage that took its services and internal communications tools offline for several hours was due to a “faulty configuration change” to its routers. It said apps are now back online after the hours-long worldwide outage. “This disruption to network traffic had a cascading effect on the way our data centres communicate, bringing our services to a halt,” added Facebook.

Some cyber security experts have a different take. Dewoskin is of the view that the outage is a response to a former Facebook employee-turned whistleblower, Frances Haugen, who spoke about safety concerns related to Facebook on CBS’ 60 Minutes Sunday night. That same whistleblower is scheduled to speak in front of Congress on Tuesday.

Haugen, a 37-year-old data scientist from Iowa, has worked for companies including Google and Pinterest but said in the CBS interview that Facebook was “substantially worse” than anything she had seen before.

“I think they’re absolutely related,” said Dewoskin. For years, US lawmakers have threatened to regulate Facebook and other social media giants to address criticisms that the platforms fuel railroad privacy issues, increase hate crimes, cause polarisation amongst communities.

What should users do?

Security experts recommend changing Facebook, Instagram and Whatsapp passwords as well as passwords of accounts that are linked to Facebook. Also, it is unsure whether users personal data were leaked during the outage.

Venkatesh Ganesh

Read Previous

Airtel & Ericsson Trials India’s first 5G network Demo

Read Next

Japan Emerges as a Strong Data Centre Hub