Fit a straight-line rule

Linear regression predicts a number by combining a baseline with weighted feature values.

The idea

Start with a baseline, then add or subtract a fixed amount for each unit of a clue. That creates a straight-line prediction.

For one feature, ŷ = wx + b. The slope w controls how much the prediction changes with x; b is the intercept. Training minimizes a regression loss.

How it works

1. Represent each example with numeric features.
2. Multiply features by coefficients and add the intercept.
3. Compare predictions with observed targets to obtain residuals.
4. Fit coefficients to minimize a loss such as residual sum of squares; then evaluate residual behavior on held-out data.

Linear prediction and ordinary least squares

The prediction is an affine function of the features. Ordinary least squares chooses coefficients that minimize squared residuals.

ŷ = Xw + b, minimize Σᵢ(yᵢ − ŷᵢ)²

Check your understanding

In ŷ = wx + b, what does w control?

• The slope / feature effect
• The number of test rows
• The class threshold
• The random seed

A common misconception

A positive coefficient means positive fitted association conditional on the other included features; it does not automatically mean changing that feature will causally change the target.

Open Linear Regression visualizer