Explainable AI Lab
Freeze one fitted model, then inspect permutation importance, PDP, ICE, and exact local model terms without pretending the explanations are the model itself.
A small nonlinear regression model is fitted locally, evaluated on held-out data, and then frozen for inspection. Learners can see model-agnostic permutation importance, average feature effects, individual conditional effects, and exact contributions of the educational model basis.
Step by step
- Fit one small nonlinear model on training rows only.
- Measure baseline MSE on held-out validation rows.
- Shuffle one feature at a time and measure validation-score degradation.
- Sweep one feature to calculate partial dependence while the model stays frozen.
- Keep individual rows separate to reveal ICE curves and interactions.
- Select one held-out row and add the exact model-term contributions back to its prediction.
Core formulas
Permutation importance
Iⱼ = MSE(permuted feature j) − MSE(baseline)A larger held-out error increase means the fitted model relied more on that feature.
Partial dependence
PDⱼ(z) = (1/n) Σᵢ f(z, xᵢ,−j)Average predictions show the model response while one feature is swept.
ICE
ICEᵢ,ⱼ(z) = f(z, xᵢ,−j)Keeping rows separate reveals heterogeneous feature effects.
When to use Explainable AI Lab
- Learning the difference between global and local model inspection.
- Seeing why feature importance is about a fitted model, not intrinsic truth about a feature.
- Detecting interactions when ICE curves do not move in parallel.