Skip to content
Armand Notue

Machine Learning · Deep Learning · Anomaly Detection

Real-Time Fraud Detection Ensemble

A stacked ensemble where autoencoder and Isolation Forest anomaly scores feed an XGBoost decision — tuned to catch fraud at a very low false-alarm rate.

Complete
TXNS44 FEATURESDETECTORSXGBOOSTP(FRAUD)AUTOENCODERISOLATION FORESTSCALED FEATURESTHRESHOLDSchematic
Problem
Card fraud is rare — under 0.2% of transactions in the benchmark data — so accuracy is meaningless, and every false alarm is friction for a real customer.
Approach
An autoencoder trained only on legitimate transactions and an Isolation Forest each produce an anomaly score; XGBoost makes the final call over 44 engineered features plus both scores.

Key results

ROC-AUC
0.9834
recall
85.71%
false-positive rate
0.16%

01

Overview

An independent project on the public ULB credit-card dataset (284,807 transactions): three complementary detectors combined into one decision, evaluated on a held-out test split.

02

Problem & motivation

Different fraud patterns need different detectors. Unsupervised anomaly scores can flag unusual behavior without depending on labels; a supervised model can learn which anomalies actually matter.

03

My role

  • Independent project — feature engineering, modeling, evaluation and write-up.

04

System architecture

  1. Stage 1:Input

    • Transaction

      30 anonymized PCA components, amount and time

  2. Stage 2:Features

    • 44 engineered features

      Amount transforms, cyclical time, PCA summary stats, z-scores

  3. Stage 3:Detectors

    • Autoencoder

      PyTorch, trained on legitimate transactions; reconstruction error

    • Isolation Forest

      Independent tree-based anomaly score

    • Scaled features

      Passed through with RobustScaler

  4. Stage 4:Decision

    • XGBoost

      Final classifier with class weighting for the imbalance

  5. Stage 5:Output

    • Fraud probability

      Compared against a decision threshold

Stacking: anomaly scores become features for the final classifier.

05

Technical implementation

  1. Feature engineering

    Amount transforms (log, square root, z-score), a cyclical encoding of time of day (sine and cosine), summary statistics across the PCA components, and z-scores for selected components — 44 features in total, scaled with a RobustScaler.

  2. An autoencoder that only knows “normal”

    A PyTorch autoencoder with a 14-dimension bottleneck is trained on legitimate transactions only. Its reconstruction error becomes an anomaly score for anything it hasn't learned to reproduce.

  3. Isolation Forest

    A tree-based outlier detector contributes a second anomaly signal that doesn't share the autoencoder's assumptions.

  4. Stacking with XGBoost

    Scaled features plus both anomaly scores feed an XGBoost classifier with class weighting for the extreme imbalance, so the supervised model learns which kinds of “unusual” are actually fraud.

06

Engineering decisions & challenges

  • Optimize for false alarms, not accuracy

    With fraud under 0.2% of transactions, a model that flags nothing is over 99.8% accurate. Evaluation centers on recall, false-positive rate, precision and ROC-AUC instead.

  • Report precision honestly

    At a 0.16% false-positive rate, precision is 47.46% — roughly one in two alerts is real fraud. Where to set that trade-off is a business decision, so it is shown rather than hidden.

  • Ensemble over a single model

    Reconstruction error, isolation depth and raw features fail in different ways; stacking lets each cover the others' blind spots.

07

Evaluation & results

Held-out test split of the public benchmark dataset.

ROC-AUC
0.9834
recallShare of fraud caught
85.71%
false-positive rateAbout 1 in 625 legitimate transactions flagged
0.16%
precision
47.46%
F1 score
0.611
mean scoring latency101 ms max, notebook benchmark
86 ms

Detection on the held-out test set

84 of 98 frauds caught while flagging 93 of 56,864 legitimate transactions — the trade-off behind 85.7% recall and 47.5% precision.

Confusion matrixTest-set transactions
Predicted legitimatePredicted fraudActual
legitimate56,771True negatives93False positivesActual
fraud14False negatives84True positives

Shade shows the count on a log scale.

Show data table
Actual \ PredictedLegitimateFraud
Legitimate56,77193
Fraud1484
Model scores0 – 1 · higher is better
  • ROC-AUC0.983
  • Recall0.857
  • F1 score0.611
  • Precision0.475
Show data table
ItemValue
ROC-AUC0.983
Recall0.857
F1 score0.611
Precision0.475

Source: Notebook confusion matrix and README results table

  • Results come from a public benchmark dataset, not a live payment system.
  • The notebook also includes a simple cost model (assuming a $500 average fraud, $10 per investigation and $5 of customer friction per false alarm). Under those assumptions it estimates about $32.8K net benefit per 100K transactions — a modeled scenario, not realized savings.
  • A packaged serving layer is planned; the repository's src/ module is marked “coming soon”.

08

From the notebook

Real figures from the project's notebook, as published in the repository. Select one to enlarge it.

09

Lessons learned

  1. On imbalanced problems, the metric is the model

    Choosing recall at a low false-positive rate over accuracy shaped every downstream decision, from features to thresholds.

  2. Unsupervised signals make strong features

    Anomaly scores give a supervised model a view of “unusual” that labels alone don't provide — in the final model, the Isolation Forest score is the second most important feature, behind only one PCA component.

10

Technologies used

  • Python
  • PyTorch
  • scikit-learn
  • XGBoost
  • pandas
  • NumPy

Want the deeper walkthrough?

I'm happy to go through the code, the trade-offs and what I'd do next.

Get in touch