Unsupervised Machine Learning from First Principles

About Course

Unsupervised Machine Learning from First Principles teaches you how to discover patterns, structure and unusual behaviour in data without relying on predefined labels.

You will move beyond simply calling machine-learning libraries. Important algorithms are explained from first principles before you implement, test and compare them using Python. You will study clustering, dimensionality reduction, anomaly detection, probabilistic models, representation learning and topic discovery.

Throughout the course, you will work with synthetic data, retail transactions, fashion images, industrial machine data and real financial complaint narratives. You will learn how to evaluate results when there is no single correct answer and how to translate discovered patterns into useful real-world insights.

By the end of the course, you will have completed five practical projects and built a portfolio demonstrating how unsupervised machine learning can be applied to business, financial services, computer vision and industrial problems.

* Understand how machines learn patterns without labelled examples
* Explain distance, similarity, density and representation from first principles
* Implement K-means and Principal Component Analysis from scratch
* Compare K-means, K-medoids, hierarchical clustering, DBSCAN, HDBSCAN, mean shift and spectral clustering
* Build soft-clustering models using fuzzy c-means and Gaussian mixture models
* Understand and implement the expectation-maximisation algorithm
* Evaluate clusters using internal, external and stability-based metrics
* Apply PCA, Kernel PCA, SVD, t-SNE, UMAP and manifold-learning techniques
* Build anomaly detectors using reconstruction error, Isolation Forest, Local Outlier Factor and One-Class SVM
* Build autoencoders for dimensionality reduction, representation learning and anomaly detection
* Discover themes in unstructured text using NMF, LDA, embeddings and clustering
* Work with messy, high-dimensional and unlabelled real-world data
* Select an appropriate algorithm based on the structure of the data
* Evaluate models without using labels during training
* Communicate discovered patterns, limitations and recommendations clearly
* Complete projects involving customer segmentation, fashion images, machine failures and financial complaints

Course Content

Unsupervised Learning from First Principles
What learning without labels means Supervised, unsupervised and semi-supervised learning The main families of unsupervised algorithms Distance, similarity and density Why feature scaling matters How to evaluate models without ground-truth labels Build: Implement common distance metrics and explore how scaling changes the apparent structure of a dataset.

Building a Supervised Benchmark
Keep your existing supervised-learning project here. Logistic regression Random forests XGBoost and LightGBM Precision, recall, ROC and auROC Test-set evaluation Ensemble models Its purpose should be clearly explained: it gives us a labelled benchmark against which we can later assess what the unsupervised methods discover.

Clustering Fundamentals
What constitutes a cluster? Centroids, medoids and within-cluster variation K-means from first principles K-means++ initialisation Choosing the number of clusters Elbow method and silhouette score External evaluation when labels happen to be available Build: Implement K-means without scikit-learn, then compare it with scikit-learn’s implementation.

Advanced and Density-Based Clustering
Why K-means cannot discover every cluster shape DBSCAN HDBSCAN Mean shift Spectral clustering K-medoids Noise points and clusters of varying density Comparing clustering algorithms fairly

Hierarchical Clustering
Agglomerative versus divisive clustering Single, complete, average and Ward linkage Distance matrices Building a hierarchy Reading and cutting dendrograms Choosing the number of clusters Computational limitations Build: Implement a simplified agglomerative clustering algorithm and compare linkage strategies on the same dataset.

Soft Clustering and Mixture Models
Hard versus probabilistic cluster assignments Fuzzy c-means Gaussian mixture models Multivariate Gaussian distributions The expectation-maximisation algorithm Covariance structures Selecting components using AIC and BIC Interpreting membership probabilities Build: Implement expectation-maximisation for a Gaussian mixture model, then compare it with K-means on overlapping clusters.

Dimensionality Reduction and Component Analysis
This would retain and reorganise your existing material: PCA from first principles Eigenvectors, eigenvalues and explained variance Scree plots Incremental, Sparse and Kernel PCA Singular value decomposition Random projections Isomap, MDS and LLE t-SNE Add UMAP as a more modern technique Choosing between visualisation and compression methods Build: Implement PCA using NumPy and SVD before comparing it with scikit-learn on MNIST or Fashion-MNIST.

Anomaly and Novelty Detection
Expand your current PCA-based anomaly-detection project with: Defining anomalies Point, contextual and collective anomalies Reconstruction-error methods Statistical thresholds Kernel density estimation Local Outlier Factor One-Class SVM Isolation Forest PCA and autoencoder anomaly detection Evaluating highly imbalanced anomaly datasets Build: Create a common evaluation framework and compare every anomaly detector on the same dataset.

Unsupervised Representation Learning
Why neural networks can learn without class labels Autoencoders from first principles Bottleneck representations Undercomplete and sparse autoencoders Denoising autoencoders Convolutional autoencoders Variational autoencoders Latent-space visualisation Representation quality and reconstruction quality Build: Construct an autoencoder in PyTorch and compare its learned representation with PCA.

Generative Models and Self-Organising Maps
Learning a data-generating distribution Variational autoencoders GAN architecture and training dynamics Generator and discriminator objectives Mode collapse and instability Self-organising maps Topology-preserving representations When these techniques remain useful Build: Train a small generative model on Fashion-MNIST and build a self-organising map for a structured tabular dataset.

End-to-End Projects
## Projects You Will Build This course is built around practical projects. You will not simply call an algorithm and accept its output. You will build important parts from first principles, compare multiple approaches, investigate where they fail and explain what your results mean in a real-world setting. ### Project 1: Clustering Algorithm Laboratory You will begin with a controlled environment where you know the true structure of the data. You will generate datasets containing: * Spherical clusters * Elongated clusters * Concentric circles * Interlocking moons * Overlapping distributions * Clusters with different densities * Noise and outliers You will then compare K-means, K-medoids, hierarchical clustering, DBSCAN, HDBSCAN, mean shift, spectral clustering, fuzzy c-means and Gaussian mixture models. You will see why an algorithm can perform well on one dataset and fail completely on another. By the end of the project, you will understand that different algorithms make different assumptions about what a cluster is. What you will build: A visual clustering laboratory that allows you to compare algorithms across different data shapes and conditions. ### Project 2: Customer Segmentation from Raw Transactions You will work with more than one million real transactions from a UK online retailer. The data contains cancellations, returns, missing customer identifiers, unusual quantities and inconsistent product information. Your first challenge will be turning this messy transactional data into something that can support meaningful analysis. You will: * Clean and validate the transaction data * Handle cancellations, returns and missing values * Create customer-level features * Calculate recency, frequency and monetary-value features * Add purchasing behaviour and product-diversity features * Handle skewed distributions and extreme customers * Reduce the number of dimensions * Compare several clustering algorithms * Evaluate the quality and stability of your clusters * Profile and name the resulting customer segments * Recommend actions for each segment You will implement K-means and its objective function yourself before comparing your implementation with scikit-learn. What you will build: A customer-segmentation system and interactive report that explains who the customer groups are, how they behave and what the business could do differently for each group. ### Project 3: Discovering Structure in Fashion Images You will work with 70,000 images of clothing from the Fashion-MNIST dataset. Each image belongs to a known category, but you will hide those categories while developing your unsupervised-learning system. Your challenge is to discover whether the images naturally organise themselves into meaningful groups without being told what any item is. You will: * Represent each image as a high-dimensional observation * Implement Principal Component Analysis from first principles * Compare PCA, Kernel PCA, t-SNE and UMAP * Measure how much information is retained after dimensionality reduction * Cluster raw pixels and reduced representations * Compare K-means, Gaussian mixtures and HDBSCAN * Build a convolutional autoencoder * Explore the representation learned inside the autoencoder * Cluster images using their learned representations * Reconstruct and denoise images * Reveal the labels only during final evaluation You will evaluate how closely the discovered clusters correspond to the actual clothing categories. What you will build: A visual product-discovery application that organises clothing images, displays representative products and identifies unusual items. ### Project 4: Detecting Machine Failures Without Training Labels You will work with industrial machine data containing measurements such as temperature, torque, rotational speed and tool wear. Failure labels exist, but you will not use them while developing your models. You will first approach the problem as it would appear in the real world: a large collection of machine observations with only a small number of unusual events. You will: * Explore normal machine operating behaviour * Establish statistical anomaly thresholds * Build anomaly detectors using PCA reconstruction error * Compare Sparse PCA and Kernel PCA * Apply Isolation Forest * Use Local Outlier Factor and One-Class SVM * Build an autoencoder anomaly detector * Compare the anomaly scores produced by different models * Reveal the failure labels during final evaluation * Measure precision, recall, PR-AUC and false-alarm rates * Select a threshold based on the cost of missed failures and false alarms You will implement reconstruction-error detection and a simplified isolation-tree mechanism from first principles. What you will build: A machine-health monitoring system that assigns anomaly scores, highlights suspicious behaviour and recommends when an observation should be investigated. ### Project 5: Discovering Emerging Problems in Financial Complaints You will work with real financial-services complaint narratives from the US Consumer Financial Protection Bureau. Unlike the previous projects, this dataset does not provide one correct set of topics for you to discover. You will need to determine whether the themes produced by your models are coherent, stable and useful. You will: * Clean and prepare real complaint narratives * Handle missing text, duplicates and repeated boilerplate * Build TF-IDF representations * Apply truncated SVD for latent semantic analysis * Discover topics using NMF and LDA * Create semantic representations using embeddings * Compare K-means and HDBSCAN * Identify representative complaints for each cluster * Write meaningful descriptions for the discovered themes * Track how complaint themes change over time * Detect unusual increases and emerging issues * Explain the limitations of conclusions drawn from complaint data You will also implement a simplified matrix-factorisation topic model and semantic similarity search. What you will build: A complaint-intelligence dashboard that presents recurring themes, representative complaints, emerging problems and changes over time. ## How the Projects Progress You will begin with controlled synthetic data where the correct structure is known. You will then progress to messy commercial transactions, high-dimensional images, rare industrial failures and unstructured financial text. By the end of the course, you will have worked with several different forms of unlabelled data and built a portfolio demonstrating clustering, dimensionality reduction, anomaly detection, representation learning and topic discovery.

Student Ratings & Reviews

No Review Yet

About Course

What Will You Learn?

Course Content

Advanced and Density-Based Clustering Why K-means cannot discover every cluster shape DBSCAN HDBSCAN Mean shift Spectral clustering K-medoids Noise points and clusters of varying density Comparing clustering algorithms fairly

Student Ratings & Reviews

Advanced and Density-Based Clustering
Why K-means cannot discover every cluster shape DBSCAN HDBSCAN Mean shift Spectral clustering K-medoids Noise points and clusters of varying density Comparing clustering algorithms fairly