Split the space with questions
A decision tree repeatedly chooses feature-based splits that make child groups more useful for prediction.
The idea
It is a chain of yes/no questions: is the value above this threshold? Then ask the next question only in the relevant branch.
At each node, candidate splits are evaluated with a criterion such as Gini impurity or entropy for classification. Recursion stops under depth or sample constraints.
How it works
- At a node, enumerate candidate feature thresholds.
- Measure how much each candidate improves the chosen criterion.
- Choose a split, create child nodes, and recurse.
- Stop or prune according to depth, leaf-size, impurity, or complexity constraints.
Gini impurity (classification)
A pure node has one class probability near 1 and impurity near 0. Other criteria such as entropy/log loss can also be used.
Gini = 1 − Σₖ pₖ²
Check your understanding
What does a tree split do?
- Partitions samples into child groups
- Applies sigmoid to every row
- Averages every feature into one
- Deletes difficult samples
Answer and explanation
Each split partitions the current samples into child nodes according to a feature rule.
A common misconception
A human-readable tree is not automatically causal, fair, or stable. Its splits describe the fitted predictor on the supplied data.
Classical ML
See how neighbors, trees, margins, and ensembles make different kinds of decisions.