NSS Labs Admits Its Test of CrowdStrike Falcon Was ‘Inaccurate’
May 25 2019NSS Labs has retracted its 2017 publicly reported and disputed test results of CrowdStrike’s Falcon endpoint security product as part of a confidential settlement reached with the security vendor over lawsuits.
The February 2017 advanced endpoint protection test report, which graded Falcon poorly, was challenged in court by CrowdStrike in a lawsuit, which alleged that the testing was incomplete and conducted using illegally obtained Falcon software, and defied CrowdStrike’s request for NSS Labs to halt the testing.
In a statement posted on its website this week, NSS Labs said that its 2017 test results of CrowdStrike Falcon were inaccurate and had been retracted.
“NSS’s testing of the CrowdStrike Falcon platform was incomplete and the product was not properly configured with prevention capabilities enabled. In addition to the results having already been acknowledged as partially incomplete, we now acknowledge they are not accurate and confirm that they do not meet our standards for publication,” NSS Labs said in the statement, which also included an apology to CrowdStrike for the “inaccurate” test results.
NSS Labs released the full AEP test report, including the flawed results of Falcon, during the 2017 RSA Conference. CrowdStrike had requested a temporary restraining order and preliminary injunction against NSS Labs to halt the report’s publication, but the court dismissed that request and the report went out.
George Kurtz, president and CEO of CrowdStrike, at that time said the tests were run using incomplete and incorrect information, and run improperly. CrowdStrike had hired NSS Labs in 2016 to perform private testing of Falcon, but later dropped the testing deal after the concerns over the quality of tests, which detected legitimate applications like Adobe and Skype as malicious, for example.
NSS Labs, however, continued to perform public tests on Falcon using software it acquired via a reseller.
The testing organization has been no stranger to controversy and conflict with security vendors. It’s currently embroiled in another lawsuit with CrowdStrike as well as other security vendors: NSS Labs in September of 2018 filed an antitrust lawsuit against CrowdStrike, ESET, and Symantec as well as the Anti-Malware Testing Standards Organization (AMTSO), over a vendor-backed testing protocol. The nonprofit ATMTSO adopted a testing protocol standard that its members had voted for and plan to adopt.
NSS Labs accused AMTSO and the three security vendors of unfairly allowing their products to be tested only by organizations that comply with the AMTSO. CrowdStrike at the time dismissed the suit as groundless, stating: “NSS is a for-profit, pay-to-play testing organization that obtains products through fraudulent means and is desperate to defend its business model from open and transparent testing.”
Related Content:
- What You Need to Know About Zero Trust Security
- MITRE Changes the Game in Security Product Testing
- NSS Labs Files Antitrust Suit Against Symantec, CrowdStrike, ESET, AMTSO
- CrowdStrike Fails In Bid To Stop NSS Labs From Publishing Test Results At RSA
Kelly Jackson Higgins is Executive Editor at DarkReading.com. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise … View Full Bio
NSS Labs Admits Its Test of CrowdStrike Falcon Was ‘Inaccurate’
May 25 2019NSS Labs has retracted its 2017 publicly reported and disputed test results of CrowdStrike’s Falcon endpoint security product as part of a confidential settlement reached with the security vendor over lawsuits.
The February 2017 advanced endpoint protection test report, which graded Falcon poorly, was challenged in court by CrowdStrike in a lawsuit, which alleged that the testing was incomplete and conducted using illegally obtained Falcon software, and defied CrowdStrike’s request for NSS Labs to halt the testing.
In a statement posted on its website this week, NSS Labs said that its 2017 test results of CrowdStrike Falcon were inaccurate and had been retracted.
“NSS’s testing of the CrowdStrike Falcon platform was incomplete and the product was not properly configured with prevention capabilities enabled. In addition to the results having already been acknowledged as partially incomplete, we now acknowledge they are not accurate and confirm that they do not meet our standards for publication,” NSS Labs said in the statement, which also included an apology to CrowdStrike for the “inaccurate” test results.
NSS Labs released the full AEP test report, including the flawed results of Falcon, during the 2017 RSA Conference. CrowdStrike had requested a temporary restraining order and preliminary injunction against NSS Labs to halt the report’s publication, but the court dismissed that request and the report went out.
George Kurtz, president and CEO of CrowdStrike, at that time said the tests were run using incomplete and incorrect information, and run improperly. CrowdStrike had hired NSS Labs in 2016 to perform private testing of Falcon, but later dropped the testing deal after the concerns over the quality of tests, which detected legitimate applications like Adobe and Skype as malicious, for example.
NSS Labs, however, continued to perform public tests on Falcon using software it acquired via a reseller.
The testing organization has been no stranger to controversy and conflict with security vendors. It’s currently embroiled in another lawsuit with CrowdStrike as well as other security vendors: NSS Labs in September of 2018 filed an antitrust lawsuit against CrowdStrike, ESET, and Symantec as well as the Anti-Malware Testing Standards Organization (AMTSO), over a vendor-backed testing protocol. The nonprofit ATMTSO adopted a testing protocol standard that its members had voted for and plan to adopt.
NSS Labs accused AMTSO and the three security vendors of unfairly allowing their products to be tested only by organizations that comply with the AMTSO. CrowdStrike at the time dismissed the suit as groundless, stating: “NSS is a for-profit, pay-to-play testing organization that obtains products through fraudulent means and is desperate to defend its business model from open and transparent testing.”
Related Content:
- What You Need to Know About Zero Trust Security
- MITRE Changes the Game in Security Product Testing
- NSS Labs Files Antitrust Suit Against Symantec, CrowdStrike, ESET, AMTSO
- CrowdStrike Fails In Bid To Stop NSS Labs From Publishing Test Results At RSA
Kelly Jackson Higgins is Executive Editor at DarkReading.com. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise … View Full Bio
