On October 10, company CEO Chris Hylen and CTO Kunal Anand set out a post mortem detailing the incident. How could this happen with a company that specializes in data and application protection?
In summary, the problem arose due to incorrect database migration from own hosting to Amazon Web Services .
In his post, Chris Heilan lists a number of mistakes made during the migration. Together, they allowed unknown to steal the admin API key to one of the accounts in production on Amazon Web Services. An investigation revealed that unauthorized access occurred back in October 2018.
The admin key gave the attacker access to a database snapshot with various information about clients who registered before September 15, 2017. The information included email addresses, hashed and salty passwords, and for a number of clients, API keys and SSL certificates provided by clients.
Chronology of failure
According to the CTO, it all started in 2017, when the company began to consider switching to the AWS ( Relational Database Service , RDS) service, because the Incapsula system was “under significant pressure due to attracting new customers and meeting their critical requirements” . The transition to cloud hosting was required to scale the business.
However, "some of the key decisions made during the AWS evaluation process combined to extract information from the database snapshot."
One of these fatal decisions is the very creation of this snapshot.
Another mistake is to create an internal computing instance with the AWS API key, which users can access from the outside.
Thus, the attacker was able to compromise the instance, steal the key and use it to access the database snapshot.
Although the data leak took place in October 2018, Imperva only learned about the hack on August 20, 2019, when a third party sent the company a set of data from its servers, demanding a reward for the bug bounty program. Imperva claims that this third party was previously unknown to it: “We compared the dump of the SQL database in the presented dataset with our snapshots - and found a match. At the moment, we can say that customer data elements are limited to WAF accounts until September 15, 2017. The databases and snapshots of our other products have not been filtered out. ”
In accordance with the GDPR law, the company duly notified law enforcement authorities and relevant regulators. A database check and damage assessment took several weeks. After that, Imperva publicly disclosed information about the incident.
Imperva emphasizes that its own product for monitoring the activity of the database Database Activity Monitoring (DAM) in 2017 did not support AWS RDS (like any other cloud hosting) and therefore was not used internally. Only in 2019, Cloud Data Security (CDS), suitable for PaaS, was developed, which is now also used for monitoring Cloud WAF.
Lessons for the future
Anand says that Imperva has taken some steps to prevent future incidents, including:
- enhanced access control;
- increasing the number of access checks for “snapshots”;
- decommissioning inactive instances (including compromised ones);
- host active computing instances for VPN by default;
- monitoring software and installing patches without delay;
- periodic change of credentials and keys;
- Improved credential management
- increasing the frequency of scanning infrastructure.
Multi-factor authentication for the AWS management console has been connected even earlier. However, according to Anand, she would not have prevented unauthorized access to the API key.
The technical director said that, thanks to improvements in internal control, it is not possible to recur today. The new Imperva system will immediately signal in case of detection of vulnerable database instances and snapshots, such as those that led to the 2018 hack. The fact is that the AWS CloudTrail and GuardDuty logging systems worked before, and they logged unauthorized activity into the logs, they simply did not signal this.
According to CTO, during the investigation of the incident, the company did not find any other vulnerabilities and does not know about any malicious activity by cybercriminals in relation to customers who were victims of a data leak.
“At the very beginning of our investigation, we immediately notified our customers so that they could make informed decisions and act in accordance with the security measures that we recommended. Thanks to these recommendations, our customers changed more than 13,000 passwords, changed more than 13,500 SSL certificates and restored more than 1,400 API keys, Anand said. “Our mission remains the same: on behalf of our customers and their users, lead the global fight to protect data and applications from cybercriminals.”
For reference. Imperva is one of the world's leading providers of web application and data protection solutions (CDN, cloud firewall, reverse proxy, protection against DDoS attacks and so on).
The company was founded in 2002, the number of employees exceeds 1000 people, annual income: $ 321.7 million (2017). Since 2011, the company's shares have been traded on the New York Stock Exchange, but in January 2019 it was completely bought out by the private investment firm Thoma Bravo, which specializes in buying up technology and software companies.
It is difficult to say how the incident will affect the image of Imperva and how much it threatens the business. Certainly, the number of customers will not grow, and the image is spoiled.
No one is safe from DevOps errors, especially in the complex business of setting up cloud instances. But who least of all expected such errors from was Imperva.
“We accept responsibility for the fact that the incident was the result of our choice, the actions that we took or did not take before, during and after the migration of the database. We recommend that all organizations take the time to fully recognize the shared responsibility for deploying and managing applications and data in Infrastructure as a Service (IaaS) solutions, ”said Imperva CTO. “You can never“ finish ”with security. Every day we must continue to work - to evaluate and improve our processes. ”