AdaBoost
Watch difficult samples gain influence and weak learners receive weighted votes.
AdaBoost trains weak learners sequentially while increasing the relative influence of misclassified samples. ModelOrigen exposes sample weights, weighted error, learner weights, votes, and the evolving boundary.
Step by step
- Give all training samples equal weight.
- Fit a weak decision stump.
- Measure its weighted error.
- Compute the learner vote weight.
- Increase emphasis on misclassified samples and normalize weights.
- Repeat and combine learner votes.
Core formulas
Learner weight
αₘ ≈ ln((1−errₘ)/errₘ)More accurate weak learners receive stronger votes in the implemented boosting logic.
Sample reweighting
wᵢ ← wᵢ · exp(αₘ · 1[misclassified])Mistakes receive more relative influence before normalization.
When to use AdaBoost
- Weak learners can be combined into a stronger classifier.
- You want to study sample reweighting directly.
- A simple boosting baseline is useful.