Chapter 13 Model-Based Clustering: Latent Structure Models

This chapter is model-based clustering. Last week we partitioned \(\mathbf{X}\) with a distance and a criterion such as \(W(C)\). Here we treat the cluster labels as missing data, put a probability model on \(\mathbf{X}\), and estimate that model. The main example is a Gaussian mixture; a hidden Markov model is the same idea when the labels form a Markov chain in time.

Model-based clustering works by fitting a mixture model to the data, with each cluster represented by one component of that mixture.

In this framework, clustering becomes equivalent to estimating a mixture model that captures the underlying distribution of the variable \(X\). Choosing the correct number of clusters, \(K\), can be guided by model selection criteria such as AIC or BIC.

A major difficulty in model-based clustering—and in mixture modeling more broadly—is estimating the model parameters. The EM (Expectation–Maximization) algorithm is the standard tool used to obtain maximum likelihood estimates of these parameters.

Although our focus here is on clustering, mixture models belong to the wider family of hierarchical models, and their applications reach well beyond clustering alone.