11.3 Baseline Predictors
Before we go to latent factors, it is useful to look at a much simpler model. Some users rate everything highly, some items are universally liked, and there is an overall average level of ratings. So a natural first prediction for a missing entry is not a neighbor or a latent factor, but a baseline.
Start with the global average of all observed ratings, \(\mu\). Then allow each user their own offset \(a_i\) and each item its own offset \(b_j\):
\[ \hat{R}_{ij} = \mu + a_i + b_j. \]
This is just a regression on dummy variables for users and items, fit only on the observed entries. We can think of a ladder of baselines:
- predict everything by the global mean \(\mu\),
- then by a user mean \(\mu + a_i\),
- then by an item mean \(\mu + b_j\),
- then by \(\mu + a_i + b_j\).
Already this removes a lot of the obvious structure in the rating matrix. The residual
\[ \tilde{R}_{ij} = R_{ij} - \mu - a_i - b_j \]
is what neighborhood methods or latent factors should be asked to explain. In that sense, collaborative filtering and matrix factorization are models for the leftover, not for the raw stars.
We usually regularize \(a_i\) and \(b_j\) as well, especially for users or items with very few ratings — the same shrinkage idea as in penalized regression. A user who has rated only one movie should not get a huge \(a_i\).