CSTAT Fall 2025 - Introduction to Bayesian Linear Regression

Title: Introduction to Bayesian Linear Regression
Instructor: Dr. Rayleigh Lei
Date: October 16, 2025

Linear regression has long been a fundamental tool across many fields for modeling relationships between variables. Traditionally, this is done using a frequentist approach, which produces point estimates for the regression coefficients and intercept based on maximum likelihood principles.

This tutorial discusses an alternative way to perform inference, i.e. applying the Bayesian framework to linear regression. In the Bayesian framework, the goal is to estimate the distribution of these parameters given the observed data. This approach not only allows for the incorporation of prior knowledge or domain expertise, but also yields results that are often easier to interpret and statistically richer. Further, recent advances in computation have made it increasingly practical to fit Bayesian models, making Bayesian linear regression accessible and effective in modern applications.

Thus, the tutorial is designed as an accessible introduction to Bayesian linear regression, with a focus on intuition and practical implementation. To ground the concepts in a real-world example, we will use the Boston Housing dataset, a classic dataset used to predict housing prices based on various socioeconomic and geographic features. Throughout the tutorial, we will discuss how to construct a Bayesian linear model, specify priors, derive the posterior distribution, and interpret the results.