Confusion Matrix and it’s error

First let’s see what confusion matrix is ?

What is a Confusion Matrix?

A Confusion matrix is the comparison summary of the predicted results and the actual results in any classification problem use case. The comparison summary is extremely necessary to determine the performance of the model after it is trained with some training data.

Confusion Matrix

There are various components that exist when we create a confusion matrix. The components are mentioned below ::

Let’s understand TP, FP, FN, TN in terms of pregnancy analogy.

True Positive(TP):

Interpretation: You predicted positive and it’s true.

You predicted that a woman is pregnant and she actually is.

True Negative(TN):

Interpretation: You predicted negative and it’s true.

You predicted that a man is not pregnant and he actually is not.

False Positive(FP): (Type 1 Error)

Interpretation: You predicted positive and it’s false.

You predicted that a man is pregnant but he actually is not.

False Negative(FN): (Type 2 Error)

Interpretation: You predicted negative and it’s false.

You predicted that a woman is not pregnant but she actually is.

What Are False Positives? (Type 1 Error)

False positives are mislabeled security alerts, indicating there is a threat when in actuality, there isn’t. These false/non-malicious alerts (SIEM events) increase noise for already over-worked security teams and can include software bugs, poorly written software, or unrecognized network traffic.

By default, most security teams are conditioned to ignore false positives. Unfortunately, this practice of ignoring security alerts — no matter how trivial they may seem — can create alert fatigue and cause your team to miss actual, important alerts related to a real/malicious cyber threats

What Are False Negatives?(Type 2 Error)

False negatives are uncaught cyber threats — overlooked by security tooling because they’re dormant, highly sophisticated (i.e. file-less or capable of lateral movement) or the security infrastructure in place lacks the technological ability to detect these attacks.

These advanced/hidden cyber threats are capable of evading prevention technologies, like next-gen firewalls, antivirus software, and endpoint detection and response (EDR) platforms trained to look for “known” attacks and malware.

Cybercrime can be anything like:

1. Stealing of personal data

2. Identity stolen

3. For stealing organizational data

4. Steal bank card details.

5. Hack emails for gaining information.

Trade off between type 1 and type 2 error is very critical in cyber security. Let’s take another example. Consider a face recognition system which is installed infront of the data warehouse which holds critical error. Consider that the manager comes and the recognition system is unable to recognize him. He tries to log in again and is allowed in.

This seems a pretty normal scenario. But let’s consider another condition. A new person comes and tries to log himself in. The recognition system makes and error and allows him in. Now this is very dangerous. An unauthorized person has made an entry. This could be very damaging to the whole company.