2.5 Linear Transformations on \(X\)
Suppose we have a linear regression model of \(Y\) on the predictors \(X\). If we scale or shift a predictor, the fitted values \(\hat{\mathbf{y}}\) and \(R^2\) remain unchanged, but the LS estimators \(\hat{\beta}\) change.
Some Examples
Scaling a predictor, i.e. \(\tilde{x}_{i2} = 2 x_{i2}\) or \(\tilde{x}_{i3} = x_{i3}/4\), does not affect \(\hat{\mathbf{y}}\) or \(R^2\).
In the
birthweightexample, the length of a baby is recorded in cm. Converting the length column to inches by the transformation \(\tilde{X}_1 = X_1/2.54\) leaves the predicted values unchanged.
These statements remain true, if we apply any linear transformation of the \(p\) predictors as long as the transformation does not change the rank of \(\mathbf{X}\).