Beyond Accuracy: Economic Performance of Machine Learning Models in Financial Fraud Detection

Financial fraud is one of the biggest operational risks for financial institutions, generating significant financial losses and destabilizing the market. While machine learning models are good at predicting, their evaluation often relies on statistical performance metrics that don't directly translate into financial impact. This research develops an evaluation framework that integrates the costs of early fraud detection with predictive effectiveness and economic criteria for decision-making. Several supervised learning models (XGBoost, neural network, random forest, decision tree, and logistic regression) were trained and tested on an unbalanced dataset of credit card transactions. To measure the potential benefit of the models for financial institutions, the savings rate and expected loss were used, along with classic metrics such as F1 score, AUC-PR, AUC-ROC, recall, and accuracy. The economic results are highly sensitive to models with similar predictive capabilities. The ensemble methods, in particular, achieved the optimal balance between fraud detection capabilities and loss reduction, while models optimized solely for accuracy resulted in higher operating costs due to false positives or undetected fraud. The results indicate that the choice of fraud detection models should not be based solely on predictive accuracy, but also on cost asymmetry and risk tolerance. The proposed framework offers practical guidance to financial institutions seeking to align operational risk management and regulatory requirements with the implementation of machine learning, enabling risk-informed decision-making.