Ashok
Ashok's Blog

Follow

Ashok's Blog

Follow

ROC Curve and AUC

Ashok's photo
Ashok
·Jan 6, 2021·
Play this article

Consider the following sample classification model

SampleModel.png

The above model is classified with few False positives and False Negatives

Think of the above model as a cancer prediction model for a moment. In that case

False positive is a healthy patient wrongly diagnosed and recommended for more tests.

False Negative is actually a cancer patient who was not diagnosed.

False negative is the worst case scenario here.

So we need to optimize the cancer detection model to make sure there are very minimal false negative scenarios.

Think of the above model as a Spam detector model for a moment. In that case

False Positive is a good email sent to a spam folder.

False Negative is a spam email sent to inbox.

This is different from the cancer deduction model. I may be fine with occasional spam mails in my inbox but I should not miss any important mail.

So a spam detector model should be optimized for very minimal False positives.

ROC curve

ROC curve is used to evaluate the performance of a classification model. It shows the performance of a classification model at all classification thresholds.

ROC curve plots two parameters:

  • True Positive Rate
  • False Positive Rate
True Positive Rate (TPR) =  TP / (TP + FN)
False Positive Rate (FPR) = FP / (FP + TN)

An ROC curve plots TPR vs FPR at different classification thresholds. Starting with all positive thresholds to all negative thresholds.

AllPositive.png

The resultant TPR vs FPR would be

ROCCurve_DifferentThreshold.png

AUC: Area Under the ROC Curve

The AUC measures the area underneath the entire ROC curve from (0,0) to (1,1).

AreaUnderCurve.png

The AUC measures performance across all possible classification thresholds. The area under the ROC curve gives an idea about the benefit of using the test for the underlying question.

AUC ranges in value from 0 to 1. A model whose predictions are 100% wrong has an AUC of 0.0; one whose predictions are 100% correct has an AUC of 1.0.

References:

Understanding ROC and AUC

An introduction to ROC analysis

 
Share this