Extra Trees
Increase tree randomness, then inspect how many randomized trees aggregate their predictions.
Extra Trees draws a node-local random cut point for each selected feature, evaluates the valid random candidates with the real impurity criterion, chooses the best valid random split, and aggregates the resulting genuinely randomized trees.
Step by step
- Choose the training set and feature-sampling rule.
- At each node, draw one random cut point inside the observed range of every selected feature.
- Evaluate the valid random candidates with the genuine impurity criterion and choose the best one.
- Grow a real randomized decision tree and repeat for many trees.
- Collect each completed tree prediction.
- Vote for classification or average for regression.
Core formulas
Impurity gain
Gain = I(parent) − weighted I(children)The randomly drawn cut-point candidates are evaluated with the same genuine impurity reduction criterion.
Aggregate
ŷ = aggregate(T₁(x), …, Tᴮ(x))The ensemble combines the randomized trees rather than trusting one tree.
When to use Extra Trees
- You want a highly randomized tree ensemble.
- Nonlinear interactions matter and scaling should be minimal.
- You want to compare bagging-style ensembles.