Python Code for Examples in Chapter 4¶

Spline Basis Functions¶

This notebook demonstrates three common ways to construct cubic spline bases in Python:

  1. A custom truncated-power (radial) cubic spline basis
  2. B-spline bases via Patsy's bs()
  3. Natural cubic spline bases via Patsy's cr()

We use the same evaluation grid $x \in [0,1]$ and the same three interior knots $\{1/6,\, 3/6,\, 5/6\}$ throughout for easy comparison.

Truncated-power cubic spline basis¶

We first define a function that returns the full design matrix for a cubic spline expressed in a truncated-power basis. The basis consists of an intercept, a linear term, and one radial basis function $R(x,z)$ for each knot $z$.

In [1]:
import numpy as np
import matplotlib.pyplot as plt
from patsy import dmatrix
In [2]:
def R_xz(x, z):
    """Radial cubic basis function centered at knot z."""
    x = np.asarray(x)
    z = np.asarray(z)

    term1 = ((z - 0.5) ** 2 - 1 / 12) * ((x - 0.5) ** 2 - 1 / 12) / 4
    term2 = (
        (np.abs(x - z) - 0.5) ** 4
        - 0.5 * (np.abs(x - z) - 0.5) ** 2
        + 7 / 240
    ) / 24

    return term1 - term2


def cubic_spline_basis(x, knots):
    """Build the design matrix H = [1, x, R(x, z1), ..., R(x, zk)]."""
    x = np.asarray(x)
    knots = np.asarray(knots)

    H0 = np.ones_like(x)
    H1 = x
    H_knots = np.column_stack([R_xz(x, knot) for knot in knots])

    return np.column_stack([H0, H1, H_knots])

When fitting a regression model the same matrix $H$ serves directly as the design matrix.

We evaluate the basis on a fine grid and form a linear combination with fixed coefficients to illustrate a typical spline curve.

In [3]:
knots = np.array([1/6, 3/6, 5/6])
x = np.linspace(0, 1, 101)

H = cubic_spline_basis(x, knots)

beta = np.array([4, -0.05, -6, 2, 10])
f = H @ beta

Plots of the basis functions and of a linear combination¶

In [4]:
fig, axes = plt.subplots(2, 3, figsize=(12, 7))
axes = axes.ravel()

labels = [
    r"$h_0(x)=1$",
    r"$h_1(x)=x$",
    rf"$h_2(x)=R(x,{knots[0]:.2f})$",
    rf"$h_3(x)=R(x,{knots[1]:.2f})$",
    rf"$h_4(x)=R(x,{knots[2]:.2f})$",
]

for j in range(5):
    axes[j].plot(x, H[:, j], linestyle="--")
    axes[j].set_title(labels[j])
    axes[j].set_xlabel("x")

axes[5].plot(x, f, linewidth=2)
axes[5].set_title(r"$f(x)=4-0.05h_1-6h_2+2h_3+10h_4$")
axes[5].set_xlabel("x")
axes[5].set_ylabel("f(x)")

plt.tight_layout()
plt.show()
No description has been provided for this image

B-spline bases with Patsy¶

In Python the function bs() from the Patsy package constructs B-spline basis functions. It is analogous to bs() from R’s splines package and is convenient for building design matrices for regression models. See the Patsy documentation on spline regression for further details.

Specifying knot locations¶

When the knot locations are known they can be passed directly via the knots argument. Below we evaluate cubic B-splines on the unit interval with interior knots at $1/6$, $3/6$ and $5/6$. The resulting design matrix contains one column for each basis function.

In [5]:
x = np.linspace(0, 1, 101)
knots = [1/6, 3/6, 5/6]

B_splines_basis1 = dmatrix(
    "bs(x, knots=knots, degree=3, include_intercept=False, "
    "lower_bound=0, upper_bound=1) - 1",
    {"x": x, "knots": knots},
    return_type="dataframe"
)

B_splines_basis1.head(6)
Out[5]:
bs(x, knots=knots, degree=3, include_intercept=False, lower_bound=0, upper_bound=1)[0] bs(x, knots=knots, degree=3, include_intercept=False, lower_bound=0, upper_bound=1)[1] bs(x, knots=knots, degree=3, include_intercept=False, lower_bound=0, upper_bound=1)[2] bs(x, knots=knots, degree=3, include_intercept=False, lower_bound=0, upper_bound=1)[3] bs(x, knots=knots, degree=3, include_intercept=False, lower_bound=0, upper_bound=1)[4] bs(x, knots=knots, degree=3, include_intercept=False, lower_bound=0, upper_bound=1)[5]
0 0.000000 0.000000 0.000000 0.0 0.0 0.0
1 0.165912 0.003490 0.000014 0.0 0.0 0.0
2 0.304896 0.013517 0.000115 0.0 0.0 0.0
3 0.418824 0.029419 0.000389 0.0 0.0 0.0
4 0.509568 0.050534 0.000922 0.0 0.0 0.0
5 0.579000 0.076200 0.001800 0.0 0.0 0.0

The trailing - 1 in the formula removes the intercept column that Patsy adds by default, so the returned columns are exactly the B-spline basis functions.

Specifying degrees of freedom¶

Instead of listing knots explicitly one can request a desired number of degrees of freedom with the df argument. For cubic B-splines Patsy places the required interior knots at suitable quantiles of the observed $x$ values. With df=4 the design matrix therefore contains four basis functions.

In [6]:
x = np.linspace(0, 1, 101)

B_splines_basis2 = dmatrix(
    "bs(x, df=4, degree=3, include_intercept=False, "
    "lower_bound=0, upper_bound=1) - 1",
    {"x": x},
    return_type="dataframe"
)

B_splines_basis2.columns = range(1, B_splines_basis2.shape[1] + 1)

B_splines_basis2.head(6)
Out[6]:
1 2 3 4
0 0.000000 0.000000 0.000000 0.0
1 0.058214 0.000592 0.000002 0.0
2 0.112912 0.002336 0.000016 0.0
3 0.164178 0.005184 0.000054 0.0
4 0.212096 0.009088 0.000128 0.0
5 0.256750 0.014000 0.000250 0.0

Illustration of the B-spline basis functions¶

Using the same three interior knots as before we rebuild the cubic B-spline basis (101 evaluation points × 6 basis functions) and plot each function to highlight its local support.

In [7]:
x = np.linspace(0, 1, 101)
knots = [1/6, 3/6, 5/6]

B_splines_basis = dmatrix(
    "bs(x, knots=knots, degree=3, include_intercept=False, "
    "lower_bound=0, upper_bound=1) - 1",
    {"x": x, "knots": knots},
    return_type="dataframe"
)

B_splines_basis.columns = range(1, B_splines_basis.shape[1] + 1)

print("Shape:", B_splines_basis.shape)
B_splines_basis.head(6)
Shape: (101, 6)
Out[7]:
1 2 3 4 5 6
0 0.000000 0.000000 0.000000 0.0 0.0 0.0
1 0.165912 0.003490 0.000014 0.0 0.0 0.0
2 0.304896 0.013517 0.000115 0.0 0.0 0.0
3 0.418824 0.029419 0.000389 0.0 0.0 0.0
4 0.509568 0.050534 0.000922 0.0 0.0 0.0
5 0.579000 0.076200 0.001800 0.0 0.0 0.0
In [8]:
fig, axes = plt.subplots(2, 4, figsize=(12, 6), sharex=True)
axes = axes.ravel()

for j, col in enumerate(B_splines_basis.columns):
    axes[j].plot(x, B_splines_basis[col], linestyle="--")
    axes[j].set_title(rf"$h_{{{col}}}(x)$")
    axes[j].set_xlabel("x")

for ax in axes[B_splines_basis.shape[1]:]:
    ax.axis("off")

plt.tight_layout()
plt.show()
No description has been provided for this image

Natural cubic splines with Patsy¶

In R, natural cubic spline bases are obtained with ns() from the splines package. The Python counterpart is Patsy’s cr() function (usually called through dmatrix). cr() produces a natural cubic regression-spline basis that is ready to be used as a design matrix. Details are available in the Patsy spline-regression documentation.

The individual basis functions generated by Patsy need not coincide with those produced by R’s ns(), because spline bases are not unique. They nevertheless span the same function space and serve the same modelling purpose.

In [9]:
x = np.linspace(0, 1, 101)
knots = [1/6, 3/6, 5/6]

NCS_basis = dmatrix(
    "cr(x, knots=knots, lower_bound=0, upper_bound=1, "
    "constraints='center') - 1",
    {"x": x, "knots": knots},
    return_type="dataframe"
)

NCS_basis.columns = range(1, NCS_basis.shape[1] + 1)

print("Shape:", NCS_basis.shape)
NCS_basis.head(6)
Shape: (101, 4)
Out[9]:
1 2 3 4
0 -0.524008 -0.660049 -0.524008 -0.087079
1 -0.426435 -0.644280 -0.501381 -0.084812
2 -0.329323 -0.628408 -0.478891 -0.082536
3 -0.233134 -0.612331 -0.456676 -0.080240
4 -0.138330 -0.595945 -0.434874 -0.077916
5 -0.045371 -0.579147 -0.413621 -0.075554
In [10]:
fig, axes = plt.subplots(2, 3, figsize=(11, 6), sharex=True)
axes = axes.ravel()

for j, col in enumerate(NCS_basis.columns):
    axes[j].plot(x, NCS_basis[col], linestyle="--")
    axes[j].set_title(rf"$h_{{{col}}}(x)$")
    axes[j].set_xlabel("x")

for ax in axes[NCS_basis.shape[1]:]:
    ax.axis("off")

plt.tight_layout()
plt.show()
No description has been provided for this image