The Birthweight Data Set Example

 

The birthweight.csv data set contains observations on 42 babies. The variables are:

 

birthweight <- read.csv("Birthweight.csv", header = TRUE)
dim(birthweight)
## [1] 42 14
head(birthweight)
##     ID Length Birthweight Headcirc Gestation smoker mage mnocig mheight mppwt
## 1 1360     56        4.55       34        44      0   20      0     162    57
## 2 1016     53        4.32       36        40      0   19      0     171    62
## 3  462     58        4.10       39        41      0   35      0     172    58
## 4 1187     53        4.07       38        44      0   20      0     174    68
## 5  553     54        3.94       37        42      0   24      0     175    66
## 6 1636     51        3.93       38        38      0   29      0     165    61
##   fage fedyrs fnocig fheight
## 1   23     10     35     179
## 2   19     12      0     183
## 3   31     16     25     185
## 4   26     14     25     189
## 5   30     12      0     184
## 6   31     16      0     180

The ID variable is not of interest, so we remove it from the data frame:

birthweight <- birthweight[, -1]

 

Fitting a Multiple Linear Regression

We begin by fitting a linear regression model that includes all predictors and examine the least-squares summary:

birthweight.mlr.full <- lm(Birthweight ~ ., data = birthweight)
summary(birthweight.mlr.full)
## 
## Call:
## lm(formula = Birthweight ~ ., data = birthweight)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.37925 -0.25558 -0.05541  0.21700  0.60329 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)   
## (Intercept) -3.812311   2.080150  -1.833  0.07713 . 
## Length       0.034152   0.031456   1.086  0.28656   
## Headcirc     0.086594   0.029489   2.937  0.00644 **
## Gestation    0.096565   0.031323   3.083  0.00447 **
## smoker      -0.243895   0.168516  -1.447  0.15854   
## mage        -0.019004   0.018537  -1.025  0.31374   
## mnocig       0.001064   0.006854   0.155  0.87770   
## mheight      0.004187   0.014153   0.296  0.76944   
## mppwt        0.010346   0.011228   0.921  0.36443   
## fage         0.007187   0.016327   0.440  0.66305   
## fedyrs       0.003220   0.030939   0.104  0.91783   
## fnocig       0.003445   0.004013   0.858  0.39769   
## fheight     -0.013293   0.009614  -1.383  0.17731   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3342 on 29 degrees of freedom
## Multiple R-squared:  0.7834, Adjusted R-squared:  0.6938 
## F-statistic: 8.742 on 12 and 29 DF,  p-value: 1.011e-06

 

The objects returned by lm and by summary(lm(...)) can be inspected with:

names(birthweight.mlr.full)
##  [1] "coefficients"  "residuals"     "effects"       "rank"         
##  [5] "fitted.values" "assign"        "qr"            "df.residual"  
##  [9] "xlevels"       "call"          "terms"         "model"
names(summary(birthweight.mlr.full))
##  [1] "call"          "terms"         "residuals"     "coefficients" 
##  [5] "aliased"       "sigma"         "df"            "r.squared"    
##  [9] "adj.r.squared" "fstatistic"    "cov.unscaled"

 

The design matrix is obtained by

design.birthweight <- model.matrix(birthweight.mlr.full)

 

The estimated regression coefficients (including the intercept) are

birthweight.mlr.full$coef
##  (Intercept)       Length     Headcirc    Gestation       smoker         mage 
## -3.812311183  0.034151922  0.086594329  0.096565435 -0.243895136 -0.019003806 
##       mnocig      mheight        mppwt         fage       fedyrs       fnocig 
##  0.001064074  0.004187315  0.010346096  0.007187428  0.003219952  0.003444674 
##      fheight 
## -0.013293205

 

Fitted values and residuals are extracted as

full.model.residuals <- birthweight.mlr.full$resid
full.model.fitted   <- birthweight.mlr.full$fitted

 

Residual standard error and \(R^2\)

The residual variance estimator is

\[ \hat{\sigma}^2 = \frac{1}{n-p-1}\sum_i r_i^2 = \frac{RSS}{n-p-1}. \]

We can compute \(\hat{\sigma}\) directly

n <- dim(birthweight)[1]
p <- dim(birthweight)[2] - 1
sqrt(sum(birthweight.mlr.full$residuals^2) / (n - p - 1))
## [1] 0.3341631

or extract it from the summary object:

summary(birthweight.mlr.full)$sigma
## [1] 0.3341631

 

The coefficient of determination is

\[ R^2 = 1 - \frac{RSS}{TSS}. \]

It can be read from the summary or computed manually:

summary(birthweight.mlr.full)$r.squared
## [1] 0.783425
1 - sum(birthweight.mlr.full$residuals^2) / (var(birthweight$Birthweight) * (n - 1))
## [1] 0.783425
1 - var(birthweight.mlr.full$residuals) / var(birthweight$Birthweight)
## [1] 0.783425

 

Invariance of \(R^2\) and Fitted Values

We verify the invariance property of \(R^2\) by centering one of the predictors:

## Center the Length variable
birthweight.new <- birthweight
birthweight.new$Length.new <- birthweight.new$Length - mean(birthweight.new$Length)

## Model with original Length
summary(lm(Birthweight ~ ., data = birthweight.new[-14]))$r.squared
## [1] 0.783425
## Model with centered Length
summary(lm(Birthweight ~ ., data = birthweight.new[-1]))$r.squared
## [1] 0.783425

 

The fitted values from the two models are essentially identical (any tiny difference is due to floating-point rounding):

sum(fitted(lm(Birthweight ~ ., data = birthweight.new[-14])) -
    fitted(lm(Birthweight ~ ., data = birthweight.new[-1])))
## [1] -8.881784e-16

 

Behavior of \(R^2\) and \(RSS\) when Predictors Are Added

We illustrate that \(R^2\) always increases (and \(RSS\) always decreases) when a new predictor is added, even if that predictor is pure noise.

birthweight.new <- birthweight
birthweight.new$randomvar <- rnorm(42)
no.var <- 0:12
par(mfrow = c(1, 2))
plot(no.var, rsq, xlab = "Number of parameters", ylab = expression(R^2), type = "n")
lines(no.var, rsq, lwd = 2, col = "orange")
plot(no.var, rss, xlab = "Number of parameters", ylab = "RSS", type = "n")
lines(no.var, rss, lwd = 2, col = "blue")

 

This demonstrates why neither \(R^2\) nor \(RSS\) should be used as a model-selection criterion. Both quantities are still useful for understanding model fit and for constructing hypothesis tests, but they are not appropriate for choosing among models.

 

Rank Deficiency

Rank deficiency occurs when \(\mathbf{X}\) is not of full column rank, so that \(\mathbf{X}^T\mathbf{X}\) is singular.

## Create a new variable: average height of both parents
birthweight.new <- birthweight
birthweight.new$aveheight <- (birthweight.new$mheight + birthweight.new$fheight) / 2

## Regress Birthweight on all predictors
summary(lm(Birthweight ~ ., data = birthweight.new))
## 
## Call:
## lm(formula = Birthweight ~ ., data = birthweight.new)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.37925 -0.25558 -0.05541  0.21700  0.60329 
## 
## Coefficients: (1 not defined because of singularities)
##              Estimate Std. Error t value Pr(>|t|)   
## (Intercept) -3.812311   2.080150  -1.833  0.07713 . 
## Length       0.034152   0.031456   1.086  0.28656   
## Headcirc     0.086594   0.029489   2.937  0.00644 **
## Gestation    0.096565   0.031323   3.083  0.00447 **
## smoker      -0.243895   0.168516  -1.447  0.15854   
## mage        -0.019004   0.018537  -1.025  0.31374   
## mnocig       0.001064   0.006854   0.155  0.87770   
## mheight      0.004187   0.014153   0.296  0.76944   
## mppwt        0.010346   0.011228   0.921  0.36443   
## fage         0.007187   0.016327   0.440  0.66305   
## fedyrs       0.003220   0.030939   0.104  0.91783   
## fnocig       0.003445   0.004013   0.858  0.39769   
## fheight     -0.013293   0.009614  -1.383  0.17731   
## aveheight          NA         NA      NA       NA   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3342 on 29 degrees of freedom
## Multiple R-squared:  0.7834, Adjusted R-squared:  0.6938 
## F-statistic: 8.742 on 12 and 29 DF,  p-value: 1.011e-06

The coefficient of aveheight is reported as NA because the new variable is an exact linear combination of mheight and fheight. Nevertheless, R still returns fitted values for the remaining coefficients, and those fitted values are valid for prediction (they coincide with the predictions obtained after simply dropping the redundant column).

The problem is easily remedied by removing one of the linearly dependent variables. Which variable to drop is a subject-matter decision (do we prefer the average height or the two individual heights?).

A more subtle situation arises when predictors are only approximately linearly dependent (i.e., highly correlated). That case is discussed later under collinearity.

 

Testing the Significance of Groups of Predictors

We use the partial \(F\) test

\[ F = \frac{(RSS_0 - RSS_{\alpha})/(p_{\alpha}-p_0)}{RSS_{\alpha}/(n-p_{\alpha})} \sim F_{p_{\alpha}-p_0,\,n-p_{\alpha}} \]

to compare the full model with a reduced model that excludes the father-related variables:

\[ \begin{cases} H_0: & Y \sim 1 + \beta_1 X_1 + \ldots + \beta_8 X_8 \\ H_{\alpha}: & Y \sim 1 + \beta_1 X_1 + \ldots + \beta_{12} X_{12}. \end{cases} \]

## Full model already stored in birthweight.mlr.full

## Reduced model
birthweight.mlr.reduced1 <- lm(Birthweight ~ Length + Headcirc + Gestation + smoker +
                                 mage + mnocig + mheight + mppwt, data = birthweight)

A single call performs the test:

anova(birthweight.mlr.reduced1, birthweight.mlr.full)
## Analysis of Variance Table
## 
## Model 1: Birthweight ~ Length + Headcirc + Gestation + smoker + mage + 
##     mnocig + mheight + mppwt
## Model 2: Birthweight ~ Length + Headcirc + Gestation + smoker + mage + 
##     mnocig + mheight + mppwt + fage + fedyrs + fnocig + fheight
##   Res.Df    RSS Df Sum of Sq      F Pr(>F)
## 1     33 3.5610                           
## 2     29 3.2383  4   0.32271 0.7225 0.5837

The observed \(F = 0.7225\) with \(p\)-value \(0.5837\) leads us to retain the reduced model; the father-related predictors may be dropped.

We can also construct the statistic by hand:

RSS0     <- deviance(birthweight.mlr.reduced1)
RSSalpha <- deviance(birthweight.mlr.full)
p0       <- 9
palpha   <- 13
n        <- 42

F <- ((RSS0 - RSSalpha) / (palpha - p0)) / (RSSalpha / (n - palpha))
F
## [1] 0.7224983
qf(0.95, palpha - p0, n - palpha)
## [1] 2.701399

The same conclusion is reached: \(F < F_{4,29}\).

 

Testing a Single Coefficient

To test the significance of Length (\(X_1\)) we compare

\[\begin{cases} &H_0: Y \sim 1 + \qquad \quad \beta_2 X_2 + \ldots + \beta_{12}X_{12} \\ & H_{\alpha}: Y \sim 1+ \beta_1 X_1+ \beta_2 X_2 + \ldots + \beta_{12} X_{12} \\ \end{cases} \]

birthweight.mlr.reduced2 <- lm(Birthweight ~ Headcirc + Gestation + smoker + mage +
                                 mnocig + mheight + mppwt + fage + fedyrs + fnocig + fheight,
                               data = birthweight)

anova(birthweight.mlr.reduced2, birthweight.mlr.full)
## Analysis of Variance Table
## 
## Model 1: Birthweight ~ Headcirc + Gestation + smoker + mage + mnocig + 
##     mheight + mppwt + fage + fedyrs + fnocig + fheight
## Model 2: Birthweight ~ Length + Headcirc + Gestation + smoker + mage + 
##     mnocig + mheight + mppwt + fage + fedyrs + fnocig + fheight
##   Res.Df    RSS Df Sum of Sq      F Pr(>F)
## 1     30 3.3699                           
## 2     29 3.2383  1   0.13162 1.1787 0.2866
## The partial-F statistic equals the square of the ordinary t-statistic
summary(birthweight.mlr.full)
## 
## Call:
## lm(formula = Birthweight ~ ., data = birthweight)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.37925 -0.25558 -0.05541  0.21700  0.60329 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)   
## (Intercept) -3.812311   2.080150  -1.833  0.07713 . 
## Length       0.034152   0.031456   1.086  0.28656   
## Headcirc     0.086594   0.029489   2.937  0.00644 **
## Gestation    0.096565   0.031323   3.083  0.00447 **
## smoker      -0.243895   0.168516  -1.447  0.15854   
## mage        -0.019004   0.018537  -1.025  0.31374   
## mnocig       0.001064   0.006854   0.155  0.87770   
## mheight      0.004187   0.014153   0.296  0.76944   
## mppwt        0.010346   0.011228   0.921  0.36443   
## fage         0.007187   0.016327   0.440  0.66305   
## fedyrs       0.003220   0.030939   0.104  0.91783   
## fnocig       0.003445   0.004013   0.858  0.39769   
## fheight     -0.013293   0.009614  -1.383  0.17731   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3342 on 29 degrees of freedom
## Multiple R-squared:  0.7834, Adjusted R-squared:  0.6938 
## F-statistic: 8.742 on 12 and 29 DF,  p-value: 1.011e-06
summary(birthweight.mlr.full)$coefficients[2, 3]^2
## [1] 1.178721

 

Overall \(F\) Test

The \(F\) statistic reported in summary(lm(...)) tests

\[\begin{cases} &H_0: Y \sim 1 \\ & H_{\alpha}: Y \sim 1+ \beta_1 X_1 + \ldots + \beta_{j-1}X_{j-1} + \beta_{j} X_{j} + \beta_{j+1} X_{j+1} + \ldots + \beta_p X_p \\ \end{cases} \]

summary(birthweight.mlr.full)$fstatistic
##     value     numdf     dendf 
##  8.741902 12.000000 29.000000

Here \(F = 8.74\) on \((12,29)\) degrees of freedom with \(p\)-value \(1.01 \times 10^{-6}\). We reject the intercept-only model.

 

Categorical Variables

The original data contain only a binary factor. For illustration we create a three-level factor from mother’s age:

birthweight.new <- birthweight
birthweight.new$mage.group <- as.factor(
  ifelse(birthweight.new$mage < 20, "a",
         ifelse(birthweight.new$mage > 30, "c", "b")))
birthweight.mlr.categorical <- lm(Birthweight ~ ., data = birthweight.new[, -6])
model.matrix(birthweight.mlr.categorical)
##    (Intercept) Length Headcirc Gestation smoker mnocig mheight mppwt fage
## 1            1     56       34        44      0      0     162    57   23
## 2            1     53       36        40      0      0     171    62   19
## 3            1     58       39        41      0      0     172    58   31
## 4            1     53       38        44      0      0     174    68   26
## 5            1     54       37        42      0      0     175    66   30
## 6            1     51       38        38      0      0     165    61   31
## 7            1     52       34        40      0      0     157    50   31
## 8            1     53       33        42      0      0     165    61   21
## 9            1     54       38        38      0      0     172    50   20
## 10           1     50       35        38      0      0     157    48   22
## 11           1     53       33        41      0      0     164    62   37
## 12           1     51       36        40      0      0     168    53   29
## 13           1     52       36        38      0      0     164    57   35
## 14           1     53       33        41      0      0     155    55   25
## 15           1     53       34        40      0      0     167    60   30
## 16           1     48       33        37      0      0     158    54   39
## 17           1     48       35        39      0      0     162    62   27
## 18           1     48       33        34      0      0     167    64   25
## 19           1     53       34        39      0      0     165    57   23
## 20           1     43       32        33      0      0     149    45   26
## 21           1     53       38        40      1      2     170    59   24
## 22           1     51       33        41      1      7     160    53   24
## 23           1     50       30        37      1      7     165    60   20
## 24           1     50       35        39      1      7     159    52   23
## 25           1     48       30        37      1      7     163    47   20
## 26           1     46       32        35      1      7     166    57   37
## 27           1     48       30        33      1      7     161    50   20
## 28           1     58       39        41      1     12     173    70   38
## 29           1     53       34        40      1     12     163    49   41
## 30           1     49       36        40      1     12     152    48   37
## 31           1     52       35        38      1     12     165    64   38
## 32           1     51       38        39      1     17     157    48   32
## 33           1     50       33        39      1     17     156    53   24
## 34           1     50       33        45      1     25     163    54   30
## 35           1     52       36        39      1     25     170    78   40
## 36           1     53       37        41      1     25     161    66   46
## 37           1     52       37        40      1     25     170    62   30
## 38           1     52       33        39      1     25     181    69   23
## 39           1     49       34        38      1     25     162    57   32
## 40           1     53       34        41      1     35     163    51   31
## 41           1     47       33        35      1     35     170    57   23
## 42           1     53       32        40      1     50     168    61   31
##    fedyrs fnocig fheight mage.groupb mage.groupc
## 1      10     35     179           1           0
## 2      12      0     183           0           0
## 3      16     25     185           0           1
## 4      14     25     189           1           0
## 5      12      0     184           1           0
## 6      16      0     180           1           0
## 7      16      0     173           1           0
## 8      10     25     185           1           0
## 9      12      7     172           0           0
## 10     14      0     179           1           0
## 11     14      0     170           1           0
## 12     16      0     181           1           0
## 13     16      0     183           0           1
## 14     14     25     183           1           0
## 15     16     25     182           1           0
## 16     10      0     171           1           0
## 17     14      0     178           1           0
## 18     12     25     175           1           0
## 19     14      2     193           0           0
## 20     16      0     169           1           0
## 21     12     12     185           1           0
## 22     16     12     176           1           0
## 23     14      0     183           0           0
## 24     14     25     200           1           0
## 25     10     35     185           1           0
## 26     14     25     173           0           1
## 27     10     35     180           1           0
## 28     14     25     180           0           1
## 29     12     50     191           0           1
## 30     12     25     170           1           0
## 31     14     50     180           1           0
## 32     12     25     169           1           0
## 33     12      7     179           1           0
## 34     16      0     183           1           0
## 35     16     50     178           1           0
## 36     16      0     175           0           1
## 37     10     25     181           1           0
## 38     16      2     181           1           0
## 39     16     50     194           0           1
## 40     16     25     185           1           0
## 41     12     50     186           1           0
## 42     16      0     173           0           1
## attr(,"assign")
##  [1]  0  1  2  3  4  5  6  7  8  9 10 11 12 12
## attr(,"contrasts")
## attr(,"contrasts")$mage.group
## [1] "contr.treatment"

R automatically expands the factor into the appropriate dummy variables; level “a” is treated as the reference. Changing the reference level (or the coding scheme) alters the coefficient estimates but leaves fitted and predicted values unchanged.

 

Mean-Response Estimation and Prediction

To illustrate the predict function we evaluate the model at the mean of each predictor:

meanvalue <- apply(birthweight[, -2], 2, mean)
x <- data.frame(t(meanvalue))
predict(birthweight.mlr.full, x)
##        1 
## 3.312857

 

Training versus Test Error

As more predictors are added, training error (RSS or MSE) steadily decreases. Test error need not follow the same pattern. We demonstrate this by a 60/40 random split:

n <- dim(birthweight)[1]
p <- dim(birthweight)[2] - 1

ntrain <- round(n * 0.6)
set.seed(598)
train.id <- sample(1:n, ntrain)

train.MSE <- rep(0, p)
test.MSE  <- rep(0, p)

train.Y <- birthweight[train.id, 2]
test.Y  <- birthweight[-train.id, 2]

for (i in 1:p) {
  var_id <- c(1:i, (p + 1))
  var_id <- var_id[var_id != 2]   # remove the response column
  myfit  <- lm(birthweight[train.id, 2] ~ ., data = birthweight[train.id, var_id])

  train.Y.pred <- myfit$fitted
  train.MSE[i] <- mean((train.Y - train.Y.pred)^2)

  test.Y.pred <- predict(myfit, newdata = birthweight[-train.id, ])
  test.MSE[i] <- mean((test.Y - test.Y.pred)^2)
}

plot(c(1, p), range(train.MSE, test.MSE), type = "n",
     xlab = "Number of variables", ylab = "MSE")
points(train.MSE, col = "blue", pch = 5)
lines(train.MSE, col = "blue")
points(test.MSE, col = "red", pch = 16)
lines(test.MSE, col = "red")

In this particular split the test MSE first declines and later rises, illustrating the classic overfitting behavior. In general we expect the training-error curve (blue) to be monotonically decreasing, while the test-error curve (red) may eventually increase.

 

Test for Outliers

We apply the Bonferroni outlier test based on externally studentized residuals:

n <- dim(birthweight)[1]
p <- length(coef(birthweight.mlr.full))
birthweight.resid <- rstudent(birthweight.mlr.full)
bonferroni_cv <- qt(0.05 / (2 * n), n - p)
birthweight.resid.sorted <- sort(abs(birthweight.resid), decreasing = TRUE)[1:10]
print(birthweight.resid.sorted)
##        2       29       34        1       28       20       42       24 
## 2.068621 1.890299 1.830714 1.682375 1.645611 1.613154 1.435773 1.411769 
##       19       39 
## 1.347956 1.324123
birthweight.outliers <- birthweight.resid.sorted[
  abs(birthweight.resid.sorted) > abs(bonferroni_cv)]
print(birthweight.outliers)
## named numeric(0)

No studentized residual exceeds the critical value, so we conclude that the data contain no outliers according to this criterion.

 

Checking Model Assumptions

Residual-versus-fitted plot:

plot(birthweight.mlr.full, which = 1)

The points scatter randomly about zero, supporting the constant-variance assumption.

Normality check:

plot(birthweight.mlr.full, which = 2)

hist(birthweight.mlr.full$residuals)

Mild departures from normality are visible.

 

     

Collinearity: Car Seat Position Data (a Faraway data set)

 

Car drivers like to adjust the seat position for their own comfort. Car designers would find it helpful to know how different drivers will position the seat depending on their size and age. Researchers at the HuMoSim laboratory at the University of Michigan collected the following data on 38 drivers:

library(faraway)
data(seatpos)
attach(seatpos)

We first fit the full model:

position.full <- lm(hipcenter ~ ., data = seatpos)
summary(position.full)
## 
## Call:
## lm(formula = hipcenter ~ ., data = seatpos)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -73.827 -22.833  -3.678  25.017  62.337 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)  
## (Intercept) 436.43213  166.57162   2.620   0.0138 *
## Age           0.77572    0.57033   1.360   0.1843  
## Weight        0.02631    0.33097   0.080   0.9372  
## HtShoes      -2.69241    9.75304  -0.276   0.7845  
## Ht            0.60134   10.12987   0.059   0.9531  
## Seated        0.53375    3.76189   0.142   0.8882  
## Arm          -1.32807    3.90020  -0.341   0.7359  
## Thigh        -1.14312    2.66002  -0.430   0.6706  
## Leg          -6.43905    4.71386  -1.366   0.1824  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 37.72 on 29 degrees of freedom
## Multiple R-squared:  0.6866, Adjusted R-squared:  0.6001 
## F-statistic:  7.94 on 8 and 29 DF,  p-value: 1.306e-05

The overall \(F\)-test is highly significant (\(p = 1.3 \times 10^{-5}\)), yet none of the individual \(t\)-tests reaches significance. This pattern is a classic warning sign of collinearity.

The correlation matrix confirms the problem:

round(cor(seatpos), digits = 2)
##             Age Weight HtShoes    Ht Seated   Arm Thigh   Leg hipcenter
## Age        1.00   0.08   -0.08 -0.09  -0.17  0.36  0.09 -0.04      0.21
## Weight     0.08   1.00    0.83  0.83   0.78  0.70  0.57  0.78     -0.64
## HtShoes   -0.08   0.83    1.00  1.00   0.93  0.75  0.72  0.91     -0.80
## Ht        -0.09   0.83    1.00  1.00   0.93  0.75  0.73  0.91     -0.80
## Seated    -0.17   0.78    0.93  0.93   1.00  0.63  0.61  0.81     -0.73
## Arm        0.36   0.70    0.75  0.75   0.63  1.00  0.67  0.75     -0.59
## Thigh      0.09   0.57    0.72  0.73   0.61  0.67  1.00  0.65     -0.59
## Leg       -0.04   0.78    0.91  0.91   0.81  0.75  0.65  1.00     -0.79
## hipcenter  0.21  -0.64   -0.80 -0.80  -0.73 -0.59 -0.59 -0.79      1.00

Many predictors are strongly correlated with one another (e.g., \(\operatorname{Corr}(\texttt{Leg},\texttt{HtShoes}) = 0.91\)).

Because the condition number is not scale-invariant, we first standardize the predictors (excluding the intercept column):

x <- model.matrix(position.full)[, -1]
x <- scale(x)          # centers and scales
eigenvalues.x <- eigen(crossprod(x))
sqrt(eigenvalues.x$values[1] / eigenvalues.x$values[8])
## [1] 59.7662

The condition number is approximately 60 (well above the conventional threshold of 30), confirming collinearity.

Variance inflation factors tell the same story:

round(vif(position.full), digits = 2)
##     Age  Weight HtShoes      Ht  Seated     Arm   Thigh     Leg 
##    2.00    3.65  307.43  333.14    8.95    4.50    2.76    6.69

The standard error of the HtShoes coefficient is inflated by a factor of roughly \(\sqrt{307} \approx 17.5\).

A practical way forward is to examine pairwise relationships and perform partial \(F\)-tests:

cor(Seated + Thigh, Ht)
## [1] 0.9389819
cor(Seated + Leg, Ht)
## [1] 0.965607
cor(Seated + Arm, Ht)
## [1] 0.9465523
position.red1 <- lm(hipcenter ~ Age + Weight + Ht + Seated, data = seatpos)
summary(position.red1)
## 
## Call:
## lm(formula = hipcenter ~ Age + Weight + Ht + Seated, data = seatpos)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -90.869 -21.163  -3.144  26.773  59.423 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)   
## (Intercept) 478.65890  159.73362   2.997  0.00515 **
## Age           0.58396    0.42573   1.372  0.17943   
## Weight       -0.01535    0.31640  -0.049  0.96159   
## Ht           -4.99025    1.64389  -3.036  0.00466 **
## Seated        2.04632    3.41283   0.600  0.55287   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 36.83 on 33 degrees of freedom
## Multiple R-squared:  0.6599, Adjusted R-squared:  0.6186 
## F-statistic: 16.01 on 4 and 33 DF,  p-value: 2.224e-07
position.red2 <- lm(hipcenter ~ Ht, data = seatpos)
summary(position.red2)
## 
## Call:
## lm(formula = hipcenter ~ Ht, data = seatpos)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -99.956 -27.850   5.656  20.883  72.066 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 556.2553    90.6704   6.135 4.59e-07 ***
## Ht           -4.2650     0.5351  -7.970 1.83e-09 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 36.37 on 36 degrees of freedom
## Multiple R-squared:  0.6383, Adjusted R-squared:  0.6282 
## F-statistic: 63.53 on 1 and 36 DF,  p-value: 1.831e-09
anova(position.red2, position.red1)
## Analysis of Variance Table
## 
## Model 1: hipcenter ~ Ht
## Model 2: hipcenter ~ Age + Weight + Ht + Seated
##   Res.Df   RSS Df Sum of Sq      F Pr(>F)
## 1     36 47616                           
## 2     33 44774  3    2841.6 0.6981 0.5599

The partial \(F\)-test indicates that the simpler model containing only Ht is adequate relative to the larger model that also includes Age, Weight and Seated.