Skip to content
Armand Notue

Data Mining · Unsupervised Learning · Analytics

Data Mining Mastery — Classic algorithms, built from scratch

Apriori, K-Means, RFM and anomaly detection implemented from scratch and applied to more than a million real online-retail transactions.

Complete

At a glance

Status
Complete
Focus
Four algorithms implemented without ML libraries · Over a million real transactions · Pareto principle checked against real revenue
Stack
  • Python
  • NumPy
  • pandas
  • Matplotlib
Problem
Library calls hide how association rules and clustering actually work — and whether their output is useful to a business.
Approach
Implemented each algorithm from first principles in Python, applied them to 1,067,371 retail transactions, and translated the patterns into merchandising and customer insights.

Key results

retail transactions
1.07M
of revenue from the top 20% of customers
54.4%
algorithms built from scratch
4

01

Overview

A hands-on data-mining project: four classic algorithms written from first principles and run on a real online-retail dataset of 1,067,371 transactions.

02

Problem & motivation

Implementing an algorithm is the fastest way to understand its assumptions. Applying it to real transactions is the fastest way to see whether its output matters.

03

My role

  • Independent project — implementation, analysis and write-up.

04

System architecture

  1. Stage 1:Data

    • 1,067,371 transactions

      Real online-retail invoices

  2. Stage 2:Algorithms

    • Apriori

      Frequent itemsets and association rules

    • K-Means

      Customer clustering

    • RFM

      Recency, frequency, monetary scoring

    • Anomaly detection

      Unusual transactions and customers

  3. Stage 3:Insights

    • Product bundles

      Items frequently bought together

    • Customer segments

      Who to retain, who to grow

    • Pareto check

      How concentrated revenue is

One dataset, four algorithms, three kinds of insight.

05

Technical implementation

  1. Association rules

    Apriori finds sets of products bought together often enough to matter and turns them into association rules — the basis for bundle recommendations.

  2. Clustering and segmentation

    K-Means and RFM scoring group customers by purchasing behavior and value.

  3. Anomaly detection

    Anomaly detection flags transactions and customers that behave unlike the rest.

06

Evaluation & results

Key findings from the notebook.

transactions analyzed
1,067,371
of revenue from the top 20% of customersPareto principle confirmed
54.4%
estimated annual bundle impactModeled estimate, not a realized result
~$124K

Revenue concentration

The Pareto check: the top fifth of customers generate more than half of revenue.

Customers vs revenueTop 20% of customersRemaining 80%

  • Share of customers

    20%80%

  • Share of revenue

    54.4%45.6%
Show data table
Top 20% of customersRemaining 80%
Share of customers20%80%
Share of revenue54.4%45.6%

Source: Project notebook (Kaggle)

  • The bundle figure is a modeled estimate from the notebook, not a measured business outcome.

07

Technologies used

  • Python
  • NumPy
  • pandas
  • Matplotlib

Want the deeper walkthrough?

I'm happy to go through the code, the trade-offs and what I'd do next.

Get in touch