From Uncertainty to Precision: Enhancing Binary Classifier Performance through Calibration - Ecole Centrale de Marseille Accéder directement au contenu
Pré-Publication, Document De Travail (Working Paper) Année : 2024

From Uncertainty to Precision: Enhancing Binary Classifier Performance through Calibration

Résumé

The assessment of binary classifier performance traditionally centers on discriminative ability using metrics, such as accuracy. However, these metrics often disregard the model's inherent uncertainty, especially when dealing with sensitive decision-making domains, such as finance or healthcare. Given that model-predicted scores are commonly seen as event probabilities, calibration is crucial for accurate interpretation. In our study, we analyze the sensitivity of various calibration measures to score distortions and introduce a refined metric, the Local Calibration Score. Comparing recalibration methods, we advocate for local regressions, emphasizing their dual role as effective recalibration tools and facilitators of smoother visualizations. We apply these findings in a real-world scenario using Random Forest classifier and regressor to predict credit default while simultaneously measuring calibration during performance optimization.
Fichier principal
Vignette du fichier
wp_2024_-_nr_04.pdf (3.04 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04464994 , version 1 (19-02-2024)

Identifiants

  • HAL Id : hal-04464994 , version 1

Citer

Agathe Fernandes Machado, Arthur Charpentier, Emmanuel Flachaire, Ewen Gallic, François Hu. From Uncertainty to Precision: Enhancing Binary Classifier Performance through Calibration. 2024. ⟨hal-04464994⟩
22 Consultations
6 Téléchargements

Partager

Gmail Facebook X LinkedIn More