Confusion Matrix, Precision, Recall & F1-Score Calculator: Free Confusion Matrix, Precision, Recall & F1-Score Calculator tool: calculate Precision, Recall, Specificity, F1-Score, Balanced Accuracy, Matthews.
The Foundations of Classification Evaluation & Confusion Matrices
In supervised machine learning, statistical diagnostics, and data science, a confusion matrix is a structured table layout that visualizes the performance of an algorithm against ground-truth labels. While binary classifiers output continuous probability scores between 0.0 and 1.0, applying a decision threshold (typically 0.50 by default) partitions every test sample into one of four mutually exclusive quadrants:
- True Positives (TP): Positive events correctly classified as positive.
- False Positives (FP): Negative events incorrectly classified as positive (Type I error, or "false alarm").
- False Negatives (FN): Positive events incorrectly classified as negative (Type II error, or "missed detection").
- True Negatives (TN): Negative events correctly classified as negative.
The Accuracy Paradox in Imbalanced Machine Learning
The most frequently misunderstood metric in machine learning is raw Accuracy:
$text{Accuracy} = frac{TP + TN}{TP + TN + FP + FN}$
In real-world applications—such as fraud detection, rare disease identification, cybersecurity intrusion detection, and search ranking—datasets exhibit severe class imbalance. If an e-commerce platform processes 100,000 transactions where only 100 are fraudulent (a 99.9% to 0.1% skew), a model that unconditionally predicts "Legitimate" for every transaction achieves an impressive-sounding 99.90% Accuracy.
Yet, this model is completely useless: its Recall is 0.0%, and all 100 fraudulent transactions slip through undetected. Data science teams must rely on metrics that are immune to label distribution bias: Precision, Recall, Balanced Accuracy, and Matthews Correlation Coefficient.
Mathematical Formulas for Diagnostic Indices
Precision / Positive Predictive Value (PPV): $text{PPV} = frac{TP}{TP + FP}$
Recall / Sensitivity / True Positive Rate (TPR): $text{TPR} = frac{TP}{TP + FN}$
Specificity / True Negative Rate (TNR): $text{TNR} = frac{TN}{TN + FP}$
Fall-out / False Positive Rate (FPR): $text{FPR} = frac{FP}{TN + FP} = 1 - text{TNR}$
Balanced Accuracy: $text{Balanced ACC} = frac{text{TPR} + text{TNR}}{2}$
F1-Score (Harmonic Mean): $F_1 = 2 times frac{text{Precision} times text{Recall}}{text{Precision} + text{Recall}} = frac{2TP}{2TP + FP + FN}$
Matthews Correlation Coefficient (MCC):
$text{MCC} = frac{(TP times TN) - (FP times FN)}{sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}$
Step-by-Step Worked Numerical Example
Consider an AI medical screening system evaluating chest X-rays for pneumonia:
- TP: 400 true pneumonia cases detected
- FP: 80 healthy patients flagged as pneumonia
- FN: 25 pneumonia patients mistakenly declared healthy
- TN: 1,495 healthy patients verified healthy
- Total Patients: 2,000
Step 1: Compute Precision and Recall:
$text{Precision} = frac{400}{400 + 80} = frac{400}{480} = mathbf{0.8333}; (83.33%)$
$text{Recall} = frac{400}{400 + 25} = frac{400}{425} = mathbf{0.9412}; (94.12%)$
Step 2: Calculate F1-Score:
$F_1 = 2 times frac{0.8333 times 0.9412}{0.8333 + 0.9412} = frac{2 times 0.7843}{1.7745} approx mathbf{0.8839}; (88.39%)$
Step 3: Calculate Matthews Correlation Coefficient (MCC):
Numerator = $(400 times 1,495) - (80 times 25) = 598,000 - 2,000 = mathbf{596,000}$
Denominator = $sqrt{(480)(425)(1,575)(1,520)} = sqrt{488,898,000,000} approx mathbf{699,212.41}$
$text{MCC} = frac{596,000}{699,212.41} approx mathbf{+0.8524}$
An MCC of +0.8524 demonstrates an exceptionally robust predictor across both pneumonia-positive and healthy cohorts, confirming the classifier has high predictive integrity far exceeding chance.
Why Matthews Correlation Coefficient (MCC) is Superior
While the F1-score is widely quoted in academic literature, it suffers from a major mathematical flaw: it completely ignores True Negatives ($TN$). If an algorithm changes its handling of negative examples, the F1-score remains unchanged.
The Matthews Correlation Coefficient (MCC), originally developed by biochemist Brian Matthews in 1975, treats the actual and predicted categories as two binary variables and calculates their Pearson correlation coefficient. MCC returns a value between $-1.0$ and $+1.0$:
- +1.0: Perfect prediction agreement ($FP = FN = 0$).
- 0.0: No better than random coin-flipping.
- -1.0: Total inverse prediction ($TP = TN = 0$).
MCC can only achieve a high score if the classifier performs well across all four quadrants of the confusion matrix in proportion to positive and negative occurrences.