Birthweight Data Set Example
The birthweight.csv data set contains observations on
42 babies. The variables are:
ID: Unique identification number of the babyLength: Length of the baby at birth in cm (\(X_1\))Birthweight: Weight of the baby at birth in kg (\(Y\))Headcirc: Head circumference of the baby at birth in cm
(\(X_2\))Gestation: Completed weeks of gestation (\(X_3\))smoker: Whether the mother smoked {0: No, 1: Yes}
(\(X_4\))mage: Mother’s age at the time of birth (\(X_5\))mnocig: Number of cigarettes the mother smoked per day
(\(X_6\))mheight: Mother’s height in cm (\(X_7\))mppwt: Mother’s pre-pregnancy weight (\(X_8\))fage: Father’s age at the time of birth (\(X_9\))fedyrs: Father’s years of education (\(X_{10}\))fnocig: Number of cigarettes the father smoked per day
(\(X_{11}\))fheight: Father’s height (\(X_{12}\))
birthweight <- read.csv("Birthweight.csv", header = TRUE)
dim(birthweight)
## [1] 42 14
head(birthweight)
## ID Length Birthweight Headcirc Gestation smoker mage mnocig mheight mppwt
## 1 1360 56 4.55 34 44 0 20 0 162 57
## 2 1016 53 4.32 36 40 0 19 0 171 62
## 3 462 58 4.10 39 41 0 35 0 172 58
## 4 1187 53 4.07 38 44 0 20 0 174 68
## 5 553 54 3.94 37 42 0 24 0 175 66
## 6 1636 51 3.93 38 38 0 29 0 165 61
## fage fedyrs fnocig fheight
## 1 23 10 35 179
## 2 19 12 0 183
## 3 31 16 25 185
## 4 26 14 25 189
## 5 30 12 0 184
## 6 31 16 0 180
The ID variable is not of interest, so we remove it from
the data frame:
birthweight <- birthweight[, -1]
We begin by fitting a linear regression model that includes all predictors and examine the least-squares summary:
birthweight.mlr.full <- lm(Birthweight ~ ., data = birthweight)
summary(birthweight.mlr.full)
##
## Call:
## lm(formula = Birthweight ~ ., data = birthweight)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.37925 -0.25558 -0.05541 0.21700 0.60329
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.812311 2.080150 -1.833 0.07713 .
## Length 0.034152 0.031456 1.086 0.28656
## Headcirc 0.086594 0.029489 2.937 0.00644 **
## Gestation 0.096565 0.031323 3.083 0.00447 **
## smoker -0.243895 0.168516 -1.447 0.15854
## mage -0.019004 0.018537 -1.025 0.31374
## mnocig 0.001064 0.006854 0.155 0.87770
## mheight 0.004187 0.014153 0.296 0.76944
## mppwt 0.010346 0.011228 0.921 0.36443
## fage 0.007187 0.016327 0.440 0.66305
## fedyrs 0.003220 0.030939 0.104 0.91783
## fnocig 0.003445 0.004013 0.858 0.39769
## fheight -0.013293 0.009614 -1.383 0.17731
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3342 on 29 degrees of freedom
## Multiple R-squared: 0.7834, Adjusted R-squared: 0.6938
## F-statistic: 8.742 on 12 and 29 DF, p-value: 1.011e-06
The objects returned by lm and by
summary(lm(...)) can be inspected with:
names(birthweight.mlr.full)
## [1] "coefficients" "residuals" "effects" "rank"
## [5] "fitted.values" "assign" "qr" "df.residual"
## [9] "xlevels" "call" "terms" "model"
names(summary(birthweight.mlr.full))
## [1] "call" "terms" "residuals" "coefficients"
## [5] "aliased" "sigma" "df" "r.squared"
## [9] "adj.r.squared" "fstatistic" "cov.unscaled"
The design matrix is obtained by
design.birthweight <- model.matrix(birthweight.mlr.full)
The estimated regression coefficients (including the intercept) are
birthweight.mlr.full$coef
## (Intercept) Length Headcirc Gestation smoker mage
## -3.812311183 0.034151922 0.086594329 0.096565435 -0.243895136 -0.019003806
## mnocig mheight mppwt fage fedyrs fnocig
## 0.001064074 0.004187315 0.010346096 0.007187428 0.003219952 0.003444674
## fheight
## -0.013293205
Fitted values and residuals are extracted as
full.model.residuals <- birthweight.mlr.full$resid
full.model.fitted <- birthweight.mlr.full$fitted
Residual standard error and \(R^2\)
The residual variance estimator is
\[ \hat{\sigma}^2 = \frac{1}{n-p-1}\sum_i r_i^2 = \frac{RSS}{n-p-1}. \]
We can compute \(\hat{\sigma}\) directly
n <- dim(birthweight)[1]
p <- dim(birthweight)[2] - 1
sqrt(sum(birthweight.mlr.full$residuals^2) / (n - p - 1))
## [1] 0.3341631
or extract it from the summary object:
summary(birthweight.mlr.full)$sigma
## [1] 0.3341631
The coefficient of determination is
\[ R^2 = 1 - \frac{RSS}{TSS}. \]
It can be read from the summary or computed manually:
summary(birthweight.mlr.full)$r.squared
## [1] 0.783425
1 - sum(birthweight.mlr.full$residuals^2) / (var(birthweight$Birthweight) * (n - 1))
## [1] 0.783425
1 - var(birthweight.mlr.full$residuals) / var(birthweight$Birthweight)
## [1] 0.783425
We verify the invariance property of \(R^2\) by centering one of the predictors:
## Center the Length variable
birthweight.new <- birthweight
birthweight.new$Length.new <- birthweight.new$Length - mean(birthweight.new$Length)
## Model with original Length
summary(lm(Birthweight ~ ., data = birthweight.new[-14]))$r.squared
## [1] 0.783425
## Model with centered Length
summary(lm(Birthweight ~ ., data = birthweight.new[-1]))$r.squared
## [1] 0.783425
The fitted values from the two models are essentially identical (any tiny difference is due to floating-point rounding):
sum(fitted(lm(Birthweight ~ ., data = birthweight.new[-14])) -
fitted(lm(Birthweight ~ ., data = birthweight.new[-1])))
## [1] -8.881784e-16
We illustrate that \(R^2\) always increases (and \(RSS\) always decreases) when a new predictor is added, even if that predictor is pure noise.
birthweight.new <- birthweight
birthweight.new$randomvar <- rnorm(42)
no.var <- 0:12
par(mfrow = c(1, 2))
plot(no.var, rsq, xlab = "Number of parameters", ylab = expression(R^2), type = "n")
lines(no.var, rsq, lwd = 2, col = "orange")
plot(no.var, rss, xlab = "Number of parameters", ylab = "RSS", type = "n")
lines(no.var, rss, lwd = 2, col = "blue")
This demonstrates why neither \(R^2\) nor \(RSS\) should be used as a model-selection criterion. Both quantities are still useful for understanding model fit and for constructing hypothesis tests, but they are not appropriate for choosing among models.
Rank deficiency occurs when \(\mathbf{X}\) is not of full column rank, so that \(\mathbf{X}^T\mathbf{X}\) is singular.
## Create a new variable: average height of both parents
birthweight.new <- birthweight
birthweight.new$aveheight <- (birthweight.new$mheight + birthweight.new$fheight) / 2
## Regress Birthweight on all predictors
summary(lm(Birthweight ~ ., data = birthweight.new))
##
## Call:
## lm(formula = Birthweight ~ ., data = birthweight.new)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.37925 -0.25558 -0.05541 0.21700 0.60329
##
## Coefficients: (1 not defined because of singularities)
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.812311 2.080150 -1.833 0.07713 .
## Length 0.034152 0.031456 1.086 0.28656
## Headcirc 0.086594 0.029489 2.937 0.00644 **
## Gestation 0.096565 0.031323 3.083 0.00447 **
## smoker -0.243895 0.168516 -1.447 0.15854
## mage -0.019004 0.018537 -1.025 0.31374
## mnocig 0.001064 0.006854 0.155 0.87770
## mheight 0.004187 0.014153 0.296 0.76944
## mppwt 0.010346 0.011228 0.921 0.36443
## fage 0.007187 0.016327 0.440 0.66305
## fedyrs 0.003220 0.030939 0.104 0.91783
## fnocig 0.003445 0.004013 0.858 0.39769
## fheight -0.013293 0.009614 -1.383 0.17731
## aveheight NA NA NA NA
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3342 on 29 degrees of freedom
## Multiple R-squared: 0.7834, Adjusted R-squared: 0.6938
## F-statistic: 8.742 on 12 and 29 DF, p-value: 1.011e-06
The coefficient of aveheight is reported as
NA because the new variable is an exact linear combination
of mheight and fheight. Nevertheless, R still
returns fitted values for the remaining coefficients, and those fitted
values are valid for prediction (they coincide with the predictions
obtained after simply dropping the redundant column).
The problem is easily remedied by removing one of the linearly dependent variables. Which variable to drop is a subject-matter decision (do we prefer the average height or the two individual heights?).
A more subtle situation arises when predictors are only approximately linearly dependent (i.e., highly correlated). That case is discussed later under collinearity.
We use the partial \(F\) test
\[ F = \frac{(RSS_0 - RSS_{\alpha})/(p_{\alpha}-p_0)}{RSS_{\alpha}/(n-p_{\alpha})} \sim F_{p_{\alpha}-p_0,\,n-p_{\alpha}} \]
to compare the full model with a reduced model that excludes the father-related variables:
\[ \begin{cases} H_0: & Y \sim 1 + \beta_1 X_1 + \ldots + \beta_8 X_8 \\ H_{\alpha}: & Y \sim 1 + \beta_1 X_1 + \ldots + \beta_{12} X_{12}. \end{cases} \]
## Full model already stored in birthweight.mlr.full
## Reduced model
birthweight.mlr.reduced1 <- lm(Birthweight ~ Length + Headcirc + Gestation + smoker +
mage + mnocig + mheight + mppwt, data = birthweight)
A single call performs the test:
anova(birthweight.mlr.reduced1, birthweight.mlr.full)
## Analysis of Variance Table
##
## Model 1: Birthweight ~ Length + Headcirc + Gestation + smoker + mage +
## mnocig + mheight + mppwt
## Model 2: Birthweight ~ Length + Headcirc + Gestation + smoker + mage +
## mnocig + mheight + mppwt + fage + fedyrs + fnocig + fheight
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 33 3.5610
## 2 29 3.2383 4 0.32271 0.7225 0.5837
The observed \(F = 0.7225\) with \(p\)-value \(0.5837\) leads us to retain the reduced model; the father-related predictors may be dropped.
We can also construct the statistic by hand:
RSS0 <- deviance(birthweight.mlr.reduced1)
RSSalpha <- deviance(birthweight.mlr.full)
p0 <- 9
palpha <- 13
n <- 42
F <- ((RSS0 - RSSalpha) / (palpha - p0)) / (RSSalpha / (n - palpha))
F
## [1] 0.7224983
qf(0.95, palpha - p0, n - palpha)
## [1] 2.701399
The same conclusion is reached: \(F < F_{4,29}\).
To test the significance of Length (\(X_1\)) we compare
\[\begin{cases} &H_0: Y \sim 1 + \qquad \quad \beta_2 X_2 + \ldots + \beta_{12}X_{12} \\ & H_{\alpha}: Y \sim 1+ \beta_1 X_1+ \beta_2 X_2 + \ldots + \beta_{12} X_{12} \\ \end{cases} \]
birthweight.mlr.reduced2 <- lm(Birthweight ~ Headcirc + Gestation + smoker + mage +
mnocig + mheight + mppwt + fage + fedyrs + fnocig + fheight,
data = birthweight)
anova(birthweight.mlr.reduced2, birthweight.mlr.full)
## Analysis of Variance Table
##
## Model 1: Birthweight ~ Headcirc + Gestation + smoker + mage + mnocig +
## mheight + mppwt + fage + fedyrs + fnocig + fheight
## Model 2: Birthweight ~ Length + Headcirc + Gestation + smoker + mage +
## mnocig + mheight + mppwt + fage + fedyrs + fnocig + fheight
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 30 3.3699
## 2 29 3.2383 1 0.13162 1.1787 0.2866
## The partial-F statistic equals the square of the ordinary t-statistic
summary(birthweight.mlr.full)
##
## Call:
## lm(formula = Birthweight ~ ., data = birthweight)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.37925 -0.25558 -0.05541 0.21700 0.60329
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.812311 2.080150 -1.833 0.07713 .
## Length 0.034152 0.031456 1.086 0.28656
## Headcirc 0.086594 0.029489 2.937 0.00644 **
## Gestation 0.096565 0.031323 3.083 0.00447 **
## smoker -0.243895 0.168516 -1.447 0.15854
## mage -0.019004 0.018537 -1.025 0.31374
## mnocig 0.001064 0.006854 0.155 0.87770
## mheight 0.004187 0.014153 0.296 0.76944
## mppwt 0.010346 0.011228 0.921 0.36443
## fage 0.007187 0.016327 0.440 0.66305
## fedyrs 0.003220 0.030939 0.104 0.91783
## fnocig 0.003445 0.004013 0.858 0.39769
## fheight -0.013293 0.009614 -1.383 0.17731
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3342 on 29 degrees of freedom
## Multiple R-squared: 0.7834, Adjusted R-squared: 0.6938
## F-statistic: 8.742 on 12 and 29 DF, p-value: 1.011e-06
summary(birthweight.mlr.full)$coefficients[2, 3]^2
## [1] 1.178721
The \(F\) statistic reported in
summary(lm(...)) tests
\[\begin{cases} &H_0: Y \sim 1 \\ & H_{\alpha}: Y \sim 1+ \beta_1 X_1 + \ldots + \beta_{j-1}X_{j-1} + \beta_{j} X_{j} + \beta_{j+1} X_{j+1} + \ldots + \beta_p X_p \\ \end{cases} \]
summary(birthweight.mlr.full)$fstatistic
## value numdf dendf
## 8.741902 12.000000 29.000000
Here \(F = 8.74\) on \((12,29)\) degrees of freedom with \(p\)-value \(1.01 \times 10^{-6}\). We reject the intercept-only model.
The original data contain only a binary factor. For illustration we create a three-level factor from mother’s age:
birthweight.new <- birthweight
birthweight.new$mage.group <- as.factor(
ifelse(birthweight.new$mage < 20, "a",
ifelse(birthweight.new$mage > 30, "c", "b")))
birthweight.mlr.categorical <- lm(Birthweight ~ ., data = birthweight.new[, -6])
model.matrix(birthweight.mlr.categorical)
## (Intercept) Length Headcirc Gestation smoker mnocig mheight mppwt fage
## 1 1 56 34 44 0 0 162 57 23
## 2 1 53 36 40 0 0 171 62 19
## 3 1 58 39 41 0 0 172 58 31
## 4 1 53 38 44 0 0 174 68 26
## 5 1 54 37 42 0 0 175 66 30
## 6 1 51 38 38 0 0 165 61 31
## 7 1 52 34 40 0 0 157 50 31
## 8 1 53 33 42 0 0 165 61 21
## 9 1 54 38 38 0 0 172 50 20
## 10 1 50 35 38 0 0 157 48 22
## 11 1 53 33 41 0 0 164 62 37
## 12 1 51 36 40 0 0 168 53 29
## 13 1 52 36 38 0 0 164 57 35
## 14 1 53 33 41 0 0 155 55 25
## 15 1 53 34 40 0 0 167 60 30
## 16 1 48 33 37 0 0 158 54 39
## 17 1 48 35 39 0 0 162 62 27
## 18 1 48 33 34 0 0 167 64 25
## 19 1 53 34 39 0 0 165 57 23
## 20 1 43 32 33 0 0 149 45 26
## 21 1 53 38 40 1 2 170 59 24
## 22 1 51 33 41 1 7 160 53 24
## 23 1 50 30 37 1 7 165 60 20
## 24 1 50 35 39 1 7 159 52 23
## 25 1 48 30 37 1 7 163 47 20
## 26 1 46 32 35 1 7 166 57 37
## 27 1 48 30 33 1 7 161 50 20
## 28 1 58 39 41 1 12 173 70 38
## 29 1 53 34 40 1 12 163 49 41
## 30 1 49 36 40 1 12 152 48 37
## 31 1 52 35 38 1 12 165 64 38
## 32 1 51 38 39 1 17 157 48 32
## 33 1 50 33 39 1 17 156 53 24
## 34 1 50 33 45 1 25 163 54 30
## 35 1 52 36 39 1 25 170 78 40
## 36 1 53 37 41 1 25 161 66 46
## 37 1 52 37 40 1 25 170 62 30
## 38 1 52 33 39 1 25 181 69 23
## 39 1 49 34 38 1 25 162 57 32
## 40 1 53 34 41 1 35 163 51 31
## 41 1 47 33 35 1 35 170 57 23
## 42 1 53 32 40 1 50 168 61 31
## fedyrs fnocig fheight mage.groupb mage.groupc
## 1 10 35 179 1 0
## 2 12 0 183 0 0
## 3 16 25 185 0 1
## 4 14 25 189 1 0
## 5 12 0 184 1 0
## 6 16 0 180 1 0
## 7 16 0 173 1 0
## 8 10 25 185 1 0
## 9 12 7 172 0 0
## 10 14 0 179 1 0
## 11 14 0 170 1 0
## 12 16 0 181 1 0
## 13 16 0 183 0 1
## 14 14 25 183 1 0
## 15 16 25 182 1 0
## 16 10 0 171 1 0
## 17 14 0 178 1 0
## 18 12 25 175 1 0
## 19 14 2 193 0 0
## 20 16 0 169 1 0
## 21 12 12 185 1 0
## 22 16 12 176 1 0
## 23 14 0 183 0 0
## 24 14 25 200 1 0
## 25 10 35 185 1 0
## 26 14 25 173 0 1
## 27 10 35 180 1 0
## 28 14 25 180 0 1
## 29 12 50 191 0 1
## 30 12 25 170 1 0
## 31 14 50 180 1 0
## 32 12 25 169 1 0
## 33 12 7 179 1 0
## 34 16 0 183 1 0
## 35 16 50 178 1 0
## 36 16 0 175 0 1
## 37 10 25 181 1 0
## 38 16 2 181 1 0
## 39 16 50 194 0 1
## 40 16 25 185 1 0
## 41 12 50 186 1 0
## 42 16 0 173 0 1
## attr(,"assign")
## [1] 0 1 2 3 4 5 6 7 8 9 10 11 12 12
## attr(,"contrasts")
## attr(,"contrasts")$mage.group
## [1] "contr.treatment"
R automatically expands the factor into the appropriate dummy variables; level “a” is treated as the reference. Changing the reference level (or the coding scheme) alters the coefficient estimates but leaves fitted and predicted values unchanged.
To illustrate the predict function we evaluate the model
at the mean of each predictor:
meanvalue <- apply(birthweight[, -2], 2, mean)
x <- data.frame(t(meanvalue))
predict(birthweight.mlr.full, x)
## 1
## 3.312857
As more predictors are added, training error (RSS or MSE) steadily decreases. Test error need not follow the same pattern. We demonstrate this by a 60/40 random split:
n <- dim(birthweight)[1]
p <- dim(birthweight)[2] - 1
ntrain <- round(n * 0.6)
set.seed(598)
train.id <- sample(1:n, ntrain)
train.MSE <- rep(0, p)
test.MSE <- rep(0, p)
train.Y <- birthweight[train.id, 2]
test.Y <- birthweight[-train.id, 2]
for (i in 1:p) {
var_id <- c(1:i, (p + 1))
var_id <- var_id[var_id != 2] # remove the response column
myfit <- lm(birthweight[train.id, 2] ~ ., data = birthweight[train.id, var_id])
train.Y.pred <- myfit$fitted
train.MSE[i] <- mean((train.Y - train.Y.pred)^2)
test.Y.pred <- predict(myfit, newdata = birthweight[-train.id, ])
test.MSE[i] <- mean((test.Y - test.Y.pred)^2)
}
plot(c(1, p), range(train.MSE, test.MSE), type = "n",
xlab = "Number of variables", ylab = "MSE")
points(train.MSE, col = "blue", pch = 5)
lines(train.MSE, col = "blue")
points(test.MSE, col = "red", pch = 16)
lines(test.MSE, col = "red")
In this particular split the test MSE first declines and later rises, illustrating the classic overfitting behavior. In general we expect the training-error curve (blue) to be monotonically decreasing, while the test-error curve (red) may eventually increase.
We apply the Bonferroni outlier test based on externally studentized residuals:
n <- dim(birthweight)[1]
p <- length(coef(birthweight.mlr.full))
birthweight.resid <- rstudent(birthweight.mlr.full)
bonferroni_cv <- qt(0.05 / (2 * n), n - p)
birthweight.resid.sorted <- sort(abs(birthweight.resid), decreasing = TRUE)[1:10]
print(birthweight.resid.sorted)
## 2 29 34 1 28 20 42 24
## 2.068621 1.890299 1.830714 1.682375 1.645611 1.613154 1.435773 1.411769
## 19 39
## 1.347956 1.324123
birthweight.outliers <- birthweight.resid.sorted[
abs(birthweight.resid.sorted) > abs(bonferroni_cv)]
print(birthweight.outliers)
## named numeric(0)
No studentized residual exceeds the critical value, so we conclude that the data contain no outliers according to this criterion.
Residual-versus-fitted plot:
plot(birthweight.mlr.full, which = 1)
The points scatter randomly about zero, supporting the constant-variance assumption.
Normality check:
plot(birthweight.mlr.full, which = 2)
hist(birthweight.mlr.full$residuals)
Mild departures from normality are visible.
Car drivers like to adjust the seat position for their own comfort. Car designers would find it helpful to know how different drivers will position the seat depending on their size and age. Researchers at the HuMoSim laboratory at the University of Michigan collected the following data on 38 drivers:
Age: Driver’s age in yearsWeight: Driver’s weight in lbsHtShoes: Height with shoes in cmHt: Height without shoes in cmSeated: Seated height in cmArm: Lower arm length in cmThigh: Thigh length in cmLeg: Lower leg length in cmhipcenter: Horizontal distance of the midpoint of the
hips from a fixed location in the car (mm)library(faraway)
data(seatpos)
attach(seatpos)
We first fit the full model:
position.full <- lm(hipcenter ~ ., data = seatpos)
summary(position.full)
##
## Call:
## lm(formula = hipcenter ~ ., data = seatpos)
##
## Residuals:
## Min 1Q Median 3Q Max
## -73.827 -22.833 -3.678 25.017 62.337
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 436.43213 166.57162 2.620 0.0138 *
## Age 0.77572 0.57033 1.360 0.1843
## Weight 0.02631 0.33097 0.080 0.9372
## HtShoes -2.69241 9.75304 -0.276 0.7845
## Ht 0.60134 10.12987 0.059 0.9531
## Seated 0.53375 3.76189 0.142 0.8882
## Arm -1.32807 3.90020 -0.341 0.7359
## Thigh -1.14312 2.66002 -0.430 0.6706
## Leg -6.43905 4.71386 -1.366 0.1824
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 37.72 on 29 degrees of freedom
## Multiple R-squared: 0.6866, Adjusted R-squared: 0.6001
## F-statistic: 7.94 on 8 and 29 DF, p-value: 1.306e-05
The overall \(F\)-test is highly significant (\(p = 1.3 \times 10^{-5}\)), yet none of the individual \(t\)-tests reaches significance. This pattern is a classic warning sign of collinearity.
The correlation matrix confirms the problem:
round(cor(seatpos), digits = 2)
## Age Weight HtShoes Ht Seated Arm Thigh Leg hipcenter
## Age 1.00 0.08 -0.08 -0.09 -0.17 0.36 0.09 -0.04 0.21
## Weight 0.08 1.00 0.83 0.83 0.78 0.70 0.57 0.78 -0.64
## HtShoes -0.08 0.83 1.00 1.00 0.93 0.75 0.72 0.91 -0.80
## Ht -0.09 0.83 1.00 1.00 0.93 0.75 0.73 0.91 -0.80
## Seated -0.17 0.78 0.93 0.93 1.00 0.63 0.61 0.81 -0.73
## Arm 0.36 0.70 0.75 0.75 0.63 1.00 0.67 0.75 -0.59
## Thigh 0.09 0.57 0.72 0.73 0.61 0.67 1.00 0.65 -0.59
## Leg -0.04 0.78 0.91 0.91 0.81 0.75 0.65 1.00 -0.79
## hipcenter 0.21 -0.64 -0.80 -0.80 -0.73 -0.59 -0.59 -0.79 1.00
Many predictors are strongly correlated with one another (e.g., \(\operatorname{Corr}(\texttt{Leg},\texttt{HtShoes}) = 0.91\)).
Because the condition number is not scale-invariant, we first standardize the predictors (excluding the intercept column):
x <- model.matrix(position.full)[, -1]
x <- scale(x) # centers and scales
eigenvalues.x <- eigen(crossprod(x))
sqrt(eigenvalues.x$values[1] / eigenvalues.x$values[8])
## [1] 59.7662
The condition number is approximately 60 (well above the conventional threshold of 30), confirming collinearity.
Variance inflation factors tell the same story:
round(vif(position.full), digits = 2)
## Age Weight HtShoes Ht Seated Arm Thigh Leg
## 2.00 3.65 307.43 333.14 8.95 4.50 2.76 6.69
The standard error of the HtShoes coefficient is
inflated by a factor of roughly \(\sqrt{307}
\approx 17.5\).
A practical way forward is to examine pairwise relationships and perform partial \(F\)-tests:
cor(Seated + Thigh, Ht)
## [1] 0.9389819
cor(Seated + Leg, Ht)
## [1] 0.965607
cor(Seated + Arm, Ht)
## [1] 0.9465523
position.red1 <- lm(hipcenter ~ Age + Weight + Ht + Seated, data = seatpos)
summary(position.red1)
##
## Call:
## lm(formula = hipcenter ~ Age + Weight + Ht + Seated, data = seatpos)
##
## Residuals:
## Min 1Q Median 3Q Max
## -90.869 -21.163 -3.144 26.773 59.423
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 478.65890 159.73362 2.997 0.00515 **
## Age 0.58396 0.42573 1.372 0.17943
## Weight -0.01535 0.31640 -0.049 0.96159
## Ht -4.99025 1.64389 -3.036 0.00466 **
## Seated 2.04632 3.41283 0.600 0.55287
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 36.83 on 33 degrees of freedom
## Multiple R-squared: 0.6599, Adjusted R-squared: 0.6186
## F-statistic: 16.01 on 4 and 33 DF, p-value: 2.224e-07
position.red2 <- lm(hipcenter ~ Ht, data = seatpos)
summary(position.red2)
##
## Call:
## lm(formula = hipcenter ~ Ht, data = seatpos)
##
## Residuals:
## Min 1Q Median 3Q Max
## -99.956 -27.850 5.656 20.883 72.066
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 556.2553 90.6704 6.135 4.59e-07 ***
## Ht -4.2650 0.5351 -7.970 1.83e-09 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 36.37 on 36 degrees of freedom
## Multiple R-squared: 0.6383, Adjusted R-squared: 0.6282
## F-statistic: 63.53 on 1 and 36 DF, p-value: 1.831e-09
anova(position.red2, position.red1)
## Analysis of Variance Table
##
## Model 1: hipcenter ~ Ht
## Model 2: hipcenter ~ Age + Weight + Ht + Seated
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 36 47616
## 2 33 44774 3 2841.6 0.6981 0.5599
The partial \(F\)-test indicates
that the simpler model containing only Ht is adequate
relative to the larger model that also includes Age,
Weight and Seated.