Facebook, Researcher Quarrel Over Instagram Hack

UPDATED. A researcher claims he was threatened by Facebook after he responsibly disclosed a series of vulnerabilities and configuration weaknesses that allowed him to gain access to sensitive information stored on Instagram servers, including source code and the details of users and employees. Facebook, on the other hand, has accused the researcher of intentionally withholding bugs and information from its team.

Wesley Wineberg, a senior security research engineer at Synack who in October earned a significant bug bounty from Microsoft for reporting a serious authentication flaw in Live.com, started analyzing Instagram after a friend of his pointed him to a potentially vulnerable server located at sensu.instagram.com.

Instagram Vulnerabilities

Vulnerability DiscoveryThe fact that a web interface on this server had been publicly accessible was reported to Facebook by Wineberg’s friend. After further analysis, on October 21, Wineberg discovered a Ruby flaw that allowed him to achieve remote code execution (RCE). According to the expert, remote code execution was possible due to two vulnerabilities: the Sensu-Admin web application running on the server contained a hardcoded Ruby “secret token,” and the host was running a version of Ruby (3.x) that was susceptible to code execution via the Ruby session cookie.

The RCE vulnerability allowed the researcher to read a configuration file containing credentials needed to access a PostgreSQL database. Accessing the database revealed roughly 60 accounts belonging to Facebook and Instagram employees. Passwords had been encrypted using bcrypt, but the expert said he quickly cracked a dozen of them that had been extremely weak (e.g. changeme, password, instagram).
Wineberg also discovered that the server had been running on Amazon’s EC2 service and a list of more than 1,400 systems had been hardcoded into the /etc/hosts file.

The researcher said he sent two separate reports to Facebook via the company’s bug bounty program to disclose the RCE flaw and the server configuration issues. He asked Facebook if he should attempt to move further to internal network systems, but he claims the social media giant simply firewalled access to the sensu.instagram.com server without replying.

While analyzing one of the configuration files stored on the server, Wineberg discovered a key pair for Amazon Web Services (AWS). He determined that the key pair had been associated with 82 different S3 buckets, which are logical storage units in AWS.

After obtaining a second key pair, he managed to gain access to the content of the buckets. The researcher discovered that they contained images uploaded by Instagram users, API keys, static content from Instagram.com, source code for the backend server, SSL certificates and private keys (including for instagram.com and *.instagram.com), email server credentials, iOS/Android app signing keys, and other sensitive information.

“To say that I had gained access to basically all of Instagram's secret key material would probably be a fair statement. With the keys I obtained, I could now easily impersonate Instagram, or impersonate any valid user or staff member. While out of scope, I would have easily been able to gain full access to any user's account, private pictures and data,” Wineberg explained in a blog post that contains the exchange of emails between him and Facebook. “It is unclear how easy it would be to use the information I gained to then compromise the underlying servers, but it definitely opened up a lot of opportunities.”

Wineberg said he sent a third vulnerability report to Facebook to notify the company of his findings.

Responsible Disclosure and Threats from Facebook

Throughout the research and reporting process Wineberg claims to have closely followed the terms of Facebook’s bug bounty program to ensure that his actions would not break any rules. Furthermore, the says he requested additional clarifications from the company, but without results.

Facebook told SecurityWeek that Wineberg went far beyond the guidelines of its bug bounty program to obtain private data from internal systems.

Facebook’s policy states the following: “If you give us reasonable time to respond to your report before making any information public, and make a good faith effort to avoid privacy violations, destruction of data, and interruption or degradation of our service during your research, we will not bring any lawsuit against you or ask law enforcement to investigate you.”

Facebook confirmed the existence of the RCE vulnerability in the sensu.instagram.com domain and promised a $2,500 reward for Wineberg and the friend who initially reported that the server was accessible.

However, the other weaknesses that allowed the expert to gain access to sensitive data were rejected, with Facebook arguing that he violated user privacy when he accessed the data. Furthermore, Wineberg claims Facebook’s CSO, Alex Stamos, contacted him via Synack’s CEO, Jay Kaplan.

“Alex informed my employer (as far as I am aware) that I had found a vulnerability, and had used it to access sensitive data. He then explained that the vulnerability I found was trivial and of little value, and at the same time said that my reporting and handling of the vulnerability submission had caused huge concern at Facebook,” Wineberg said. “Alex then stated that he did not want to have to get Facebook's legal team involved, but that he wasn't sure if this was something he needed to go to law enforcement over.”

Stamos allegedly attempted to convince the researcher and his employer to keep the existence of the security holes private and delete all data obtained from Instagram systems.

Facebook, on the other hand, said the claims are false and that the researcher was never told that he could not publish his findings. The social media giant said it only asked the expert not to disclose the non-public information he accessed.

In a later statement posted on Facebook, Stamos said he contacted Synack's Jay Kaplan because he believed the researcher was working on the company's behalf. Synack later clarified that Wineberg conducted the research in his free time.

“I told Jay that we couldn't allow Wes to set a precedent that anybody can exfiltrate unnecessary amounts of data and call it a part of legitimate bug research, and that I wanted to keep this out of the hands of the lawyers on both sides,” Stamos said. “I did not threaten legal action against Synack or Wes nor did I ask for Wes to be fired. I did say that Wes's behavior reflected poorly on him and on Synack, and that it was in our common best interests to focus on the legitimate RCE report and not the unnecessary pivot into S3 and downloading of data.”

“In my opinion, the best course of action was to simply be transparent with all of my findings and interactions. I am not looking to shame any individuals or companies, but I do believe that my treatment in this situation was completely inappropriate,” Wineberg said.

“I continue to hope that security research will be given appropriate recognition and legal protections. In the meantime, I believe that it's the infosec community's job to lead by example. I don't think that threatening security researchers should ever be acceptable, and I believe that as a community we are better than that,” the expert added. “I don't need this write-up to act as a warning to other researchers; everyone is already aware of the risks that come with performing research. Instead, I hope that this write-up shows how far we still need to go as a community.”

Impact of Vulnerability Disclosure

The researcher believes his disclosure of the flaws should not pose any risk considering that Facebook addressed the RCE issue and closed access to the vulnerable server. Wineberg told SecurityWeek he is not disclosing any of the AWS keys or credentials, and that he deleted the information downloaded from the exposed S3 buckets at Facebook’s request.

The expert says it’s unclear what Facebook has done to resolve the other vulnerabilities.

“Weak logins are something that would be trivial to resolve, but my third issue regarding the AWS credentials is a more complicated item. To prevent continued access Facebook could change the AWS credentials, but really the whole architecture was a reflection of poor security practice,” Wineberg said via email. “Access segmentation, encryption of private keys, and other measures should all be in place to have a proper defense-in-depth setup. Further, I would hope that Facebook has since investigated whether these systems have been compromised in the past using a similar attack vector.”

Stamos said the bug has been fixed and the affected keys have been rotated. Furthermore, Facebook's CSO says there is no evidence that user data has been accessed by Wineberg or anyone else.

“We will be looking at our documentation and the operation of our program. We successfully handle hundreds of reports per day, but I don't think we triaged the reports on this issue quickly enough. We will also look at making our policies more explicit and will be working to make sure we are clearer about what we consider ethical behavior,” Stamos said.

Statement from Facebook and Synack

“We are strong advocates of the security researcher community and have built positive relationships with thousands of people through our bug bounty program. These interactions must include trust, however, and that includes reporting the details of bugs that are found and not using them to access private information in an unauthorized manner. In this case, the researcher intentionally withheld bugs and information from our team and went far beyond the guidelines of our program to pull private, non-user data from internal systems.

We paid him for his initial bug report based on the quality, even though he was not the first to report it, but we didn't pay for the subsequent information that he had withheld. At no point did we say he could not publish his findings―we asked that he refrain from disclosing the non-public information he accessed in violation of our program guidelines. We remain firmly committed to paying for high quality research and helping the community learn from researchers' hard work.”

Synack has provided the following statement, noting that Wineberg conducted the research in his free time:

“Synack values customer privacy and always responsibly and confidentially reports vulnerabilities directly to its customers. Facebook is not a Synack customer. Wes Wineberg, a well-respected independent contractor, was not acting on behalf of Synack. Synack believes ethical researchers across the world should be empowered to find, and responsibly disclose, vulnerabilities so that consumers and users are more secure.”