2.1 Multiple Linear Regression (MLR) Model
The model formulation is given by:
Multiple Linear Regression Model
\[ Y = \beta_0 + \beta_1 X_1 + \ldots + \beta_p X_p + \varepsilon \] where
\(\beta_0\) is the intercept
\(\beta_j\) is the regression coefficient associated with predictor \(X_j\)
\(\varepsilon\) is the error term
The usual assumptions for the error terms are \(\varepsilon \sim IID \bigl(0, \sigma^2 \mathbf{I} \bigr)\)
Given the training data \(\bigl\{ x_{i1},\; x_{i2}, \ldots, x_{ip};\, y_i\bigr\}_{i=1}^{n}\), we can rewrite the model as \[ y_i = \beta_0 + \beta_1 x_{i1} + \ldots + \beta_p x_{ip} + \varepsilon_i,\,\, i=1, \ldots, n\] where \(n\) is the sample size. Using the following matrix representation for the response and the features: \[ \mathbf{y}=\left(\begin{array}{c} y_{1} \\ y_{2} \\ \vdots \\ y_{n} \end{array}\right) \quad \mathbf{\varepsilon}=\left(\begin{array}{c} \varepsilon_{1} \\ \varepsilon_{2} \\ \vdots \\ \varepsilon_{n} \end{array}\right) \quad \mathbf{\beta}=\left(\begin{array}{c} \beta_{0} \\ \beta_{1} \\ \vdots \\ \beta_p \end{array}\right)\quad \mathbf{X}=\left(\begin{array}{cccc} 1 & x_{11} & \cdots & x_{1p} \\ 1 & x_{21} & \cdots & x_{2p} \\ \vdots & \vdots & \ddots & \vdots\\ 1 & x_{n1} & \cdots & x_{np} \\ \end{array} \right) \] the model equation can be written in as \[\left(\begin{array}{c} y_{1} \\ y_{2} \\ \vdots \\ y_{n} \end{array}\right) = \left(\begin{array}{ccccc} 1&x_{11}& \cdots &x_{1p} \\ 1& x_{21}& \cdots&x_{2p} \\ \vdots & \vdots & \ddots & \vdots\\ 1 & x_{n1} & \cdots & x_{np} \\ \end{array} \right) \left(\begin{array}{c} \beta_{0} \\ \beta_{1} \\ \vdots \\ \beta_p \end{array}\right)\quad + \left(\begin{array}{c} \varepsilon_{1} \\ \varepsilon_{2} \\ \vdots \\ \varepsilon_{n} \end{array}\right) \] which leads to the concise form
\[ \begin{array}{cccccc} \mathbf{y}_{n\times 1} &=&\mathbf{X}_{n\times (p+1)} &\mathbf{\beta}_{(p+1)\times 1}&+&\mathbf{\varepsilon}_{n\times 1}\\ \uparrow & & \uparrow & \uparrow & & \uparrow\\ \text{Response} &&\text{Design} & \text{Coefficients} & & \text{Error}\\ &&\text{Matrix} & && \text{Term}\\ \end{array} \] where \(n\) is the sample size, and \(p+1\) is number of columns of \(\mathbf{X}\) (the \(p\) predictors plus the intercept).