2.5 Linear Transformations on \(X\)

Suppose we have a linear regression model of \(Y\) on the predictors \(X\). If we scale or shift a predictor, the fitted values \(\hat{\mathbf{y}}\) and \(R^2\) remain unchanged, but the LS estimators \(\hat{\beta}\) change.

Some Examples

  • Scaling a predictor, i.e. \(\tilde{x}_{i2} = 2 x_{i2}\) or \(\tilde{x}_{i3} = x_{i3}/4\), does not affect \(\hat{\mathbf{y}}\) or \(R^2\).

  • In the birthweight example, the length of a baby is recorded in cm. Converting the length column to inches by the transformation \(\tilde{X}_1 = X_1/2.54\) leaves the predicted values unchanged.

These statements remain true, if we apply any linear transformation of the \(p\) predictors as long as the transformation does not change the rank of \(\mathbf{X}\).