Simple Linear Regression
The Model
Simple linear regression models a dependent variable (y
) using an independent variable (x
):
y = \beta_0 + \beta_1 x + \varepsilon, \qquad E(y \mid x) = \beta_0 + \beta_1 x.
The coefficient\beta_1
is the marginal effect ofx
- ony
-
the average change iny
whenx
increases by one unit.
Least Squares Estimation
We chooseb_0, b_1
that minimize the sum of the squares of the residuals\sum (y_i - \hat{y}_i)^2
. The solution:
b_1 = \frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sum (x_i-\bar{x})^2} = \frac{s_{xy}}{s_x^2}, \qquad b_0 = \bar{y} - b_1 \bar{x}.
The quality of the fit is measured by the coefficient of determination:
R^2 = \frac{SSR}{SST} = 1 - \frac{SSE}{SST} \in [0,1].
Try It Yourself
Adjust the true slope and the noise, resample, and observe how the estimated line (orange) approaches the true line (dotted), as well as the coefficients and theR^2
.
Doing It in R
The same calculation using the `lm()
` function—the building block of all econometrics:
set.seed(1)
n <- 40
x <- runif(n, 0, 10)
y <- 2 + 1.2 * x + rnorm(n, sd = 1.5)
modele <- lm(y ~ x)
summary(modele) # coefficients, R², tests
plot(x, y); abline(modele, col = "orange", lwd = 2)This block displays the code. To allow students to run it online (without installing R):
- Install the extension once:
quarto add coatless-templates/quarto-webr
; 2. Addwebr: {}
andfilters: [webr]
to the header of this page; 3. Replace the closing tag`
with r`par
<x id="4"/>
.
The same principle applies with Pyodide (quarto add coatless-templates/quarto-pyodide
), which allows you to run Python online.
Slides & Video
The corresponding chapter of the statistics course includes slides (downloadable as PDFs) and a short video. They will be embedded here—for example:
slides:
chapitre12.pdfvideo: YouTube link embedded via
.