Ask the nearby examples

KNN predicts from the labels or values of nearby training samples instead of fitting one global equation.

The idea

If you enter a new neighborhood, nearby houses can tell you more about typical prices there than houses across the city.

Choose a distance metric, find the k closest stored training points, then aggregate their labels or values. Scaling matters because distance depends on feature units.

How it works

1. Put the query and training examples into the same feature representation.
2. Compute distance from the query to candidate training points.
3. Select the k closest points.
4. Aggregate their labels/values using uniform or distance-based weighting.

A common distance

Minkowski distance gives Manhattan distance at p=1 and Euclidean distance at p=2. The useful metric depends on the representation and problem.

d(x,z) = (Σⱼ |xⱼ − zⱼ|ᵖ)^(1/p)

Check your understanding

If k becomes larger, what changes directly?

• More neighbors contribute to the prediction
• The model becomes a decision tree
• Features disappear
• The train/test split becomes 50/50

A common misconception

KNN is not ‘training-free’ in the practical sense: representation, scaling, indexing, metric, k, and stored data all determine behavior.

Open KNN visualizer