Find a separating margin

A linear SVM looks for a separating boundary with a large margin, with the closest influential points acting as support vectors.

The idea

Imagine drawing a lane between two groups of points and trying to keep the lane as wide as possible without ignoring costly mistakes.

For a linear SVM, the decision function is based on w·x + b. The optimization balances margin size and violations through the regularization parameter C.

How it works

1. Represent samples in feature space and scale features where appropriate.
2. Choose a linear or kernelized similarity representation.
3. Optimize a maximum-margin objective with penalties for violations.
4. Predict from the sign/value of the decision function; calibrate probabilities separately if needed.

Linear soft-margin intuition

The first term favors a wider margin; slack ξ allows violations; C controls how strongly those violations are penalized.

min ½||w||² + C Σᵢ ξᵢ subject to yᵢ(w·xᵢ+b) ≥ 1−ξᵢ

Check your understanding

Which samples most directly determine the SVM margin?

• Support vectors
• Only the easiest samples
• Only test samples
• Randomly generated samples

A common misconception

‘Maximum margin’ does not mean zero training errors. Soft-margin SVM intentionally permits violations when that improves the regularized objective.

Open SVM visualizer