Million-dollar problems – Spoiler: it’s not even a problem.

” Fairy tales for adults: from the boy who cried wolf to alert fatigue “

Once upon a time, there was a young shepherd, tired of his monotonous life in the pastures, who decided to amuse himself by pretending that a wolf was attacking his flock. He ran to the village shouting: “Wolf! Wolf!” The villagers rushed to his rescue, but discovered it was a false alarm and that the boy was making fun of them. A few days later, he repeated the joke, with the same result.

But one evening, a real wolf attacked the sheep. The boy ran to the village shouting desperately for help, but the villagers, thinking it was another joke, ignored him. Unfortunately, the wolf killed many of the boy’s sheep, and there was no one to help him.

This story, found in many cultures, not only teaches the consequences of lying, but also illustrates the concept of false positives and the alert fatigue they cause. Unlike the boy’s pranks, security false positives result from a lack of detailed information or sufficient confirmations.

The dangers of false positives in modern industries

From finance to IT security, almost every sector is confronted with false positives. These erroneous alerts cause financial damage and harm to companies’ reputations, which lose tens of thousands of work hours every year.

Many think it is difficult to tackle false positives. This is the case if the targeted or underlying systems, the anomalies or the malicious attempts are not properly understood. False positives not only hinder the improvement of vulnerability reports and incident reports, but also disrupt legitimate operations, real customer interactions and valid transactions.

Origins and implications of false positives 

The terms “ false positives“, “true positives“, “false negatives” and “true negatives” come from statistical hypothesis testing and binary classification, commonly used in fields such as medicine, statistics and security operations. In the context of  machine learning , these terms evaluate the accuracy of tests and decisions. Here we will focus on their importance in information security and incident response .

Understanding the key terms

• True Positive (TP): this term refers to a correct positive prediction or result. When your scanning tool flags an SQL injection vulnerability and you or the tool itself can confirm that the result is real and that the vulnerability exists, this result can be classified as a true positive.
  
  You or the security tool/service you used report something, and that vulnerability is indeed present. You may be wondering what “as long as I can confirm…” means. Here the problem arises: it takes a lot of human resources or technical capabilities from the tool to confirm the existence of the vulnerability.
  
  A clever formula or assumption can be applied in this area: if a vulnerability exists, it must be exploitable. If it is not, it is probably not a vulnerability

Does this remind you of  Aristotle ? You’re right! These are very basic logical rules of human intelligence. Sometimes, humans can be confused.

For such a basic logical process, why do many tools and services fail? The answer is simple: to reach this conclusion as a human, you have to spend time on it (that is precisely one of the problems), and tools and services have to know the internal workings of systems very well.

• True negative (TN):  a true negative represents a correct negative prediction or result. It indicates the correct identification of the absence of a relationship or condition that is truly not present in the data.
  
  For example, if you or a tool or service you used, after a security check, did not find an SQL injection vulnerability on the website, and the website indeed does not have this problem, this finding can be considered a true negative. You evaluated a system with a negative rating for the existence of a vulnerability, and it is accurate.
  
• False positive (FP): a false positive occurs when a test wrongly indicates the presence of a condition that is not actually present; when the test wrongly gives a positive result. Recall our first example: you or the tool/service you used reported an SQL injection vulnerability on your website. In reality, this is not true.
  
  To increase the severity of the example, let’s say your website has no database connection or the database behind the application is not a relational database. You can easily flag this result as a false positive. Basically, we are saying the result is false.
  
• False negative (FN):  a false negative is a test result that wrongly indicates the absence of a condition that actually exists. While false positives can be time-consuming, tiring and annoying, false negatives can be very risky and devastating. Imagine you ask your security team to scan your new web service for SQL injection vulnerabilities.
  
  After some time, the staff member comes back to you and says they did their best but found no SQL injection vulnerability in your web service. However, if one really does exist, marking a finding as “Not found” or negative can be misleading. This situation can be classified as a false negative. Again, while false positives require a lot of resources, time and patience, false negatives, or marking a system as healthy when a vulnerability exists, can be very dangerous.

The real damage caused by false positives

False positives can be extremely costly for organizations.  Investigating false positives can require nearly 10,000 hours of work per year, at a cost of around $500,000

The impact of this fraud goes beyond information security. In e-commerce, false positives, or false declines, represent a  revenue loss of $443 billion per year , which far exceeds the actual costs of credit card fraud. Incorrectly blocking legitimate transactions can  potentially cost businesses up to $386 billion per year.

Moreover, false positives harm the company’s reputation. They lead to poor customer experiences, with legitimate customers turned away or transactions blocked. This can lead to customer loss,  with 38% of consumers saying they would switch merchants after a false decline .

Mitigating false positives: the cybersecurity team’s approach

False positives in security alerts are a major challenge for security analysts

Detailed explanations of the steps to follow to handle FPs are as follows :

• Integration with Threat Intelligence : Integration with SIEM (Security Information and Event Management) solutions, which collect logs and data from various sources on the network. This integration enhances SIEM capabilities by providing contextual information on potential threats, allowing analysts to understand attackers’ tactics, techniques and procedures (TTPs). This reduces false positives and helps analysts focus on critical incidents.
  
• User-driven scoring approach:  prioritizing alerts according to their severity and relevance, assigning scores to ensure analysts focus on the most important threats, thereby reducing false-positive noise.
  
• Validation and whitelisting:  validate indicators of compromise (IoCs)  and compare them against a whitelist. With  more than 43 million  whitelist policies, known benign entities are filtered out, thus avoiding false positives. Analysts can also check the whitelist status of any IP address, hash or domain.

By implementing these measures, the occurrence of false positives is effectively reduced, allowing security teams to operate more efficiently and focus on real threats.