Training source
Full IEEE-CIS training population
RETROSPECTIVE IEEE-CIS RESEARCH
Measure uncertainty.
Publish the evidence.
A versioned calibration study with rolling development windows, a separate calibration period and a final chronological evaluation. All results describe the licensed historical competition dataset.
The dashboard's historical v2 model remains an uncalibrated ranking model. This new model has its own version and evaluation; its probability mapping must not be applied to the old dashboard scores.
Loading verified research evidence…
Development windows
Calibration method selected before final evaluation
Selected mapping
Final chronological evaluation
Each method uses the same final records. The constant baseline is fitted from the earlier calibration period. Lower Brier score, log loss and calibration error are better; higher ROC-AUC and average precision are better.
| Method | ROC-AUC | Average precision | Brier score | Log loss | Calibration error |
|---|
Reliability of the selected mapping
Average predicted probability and observed fraud prevalence within populated, fixed probability bins. The diagonal represents perfect empirical calibration. Sparse bins and temporal dependence limit inference.
Research boundaries
This page serves committed aggregate evidence. It performs no model training, BigQuery queries or transaction scoring.
Reproduce and inspect
Calibration evidence JSON · Dated BigQuery reconciliation · Repository & reproducible source
- Training transaction SHA-256
- Publication artifact SHA-256