Explainability Guide¶

Endgame provides a unified explainability layer that works with any sklearn-compatible model. Techniques range from one-line auto-explanation to detailed SHAP, LIME, partial dependence, interaction detection, and counterfactual generation.

Import convention: import endgame as eg

The `explain()` One-Liner¶

explain() is the fastest entry point. It inspects the model type, selects the most appropriate explanation method automatically, and returns a summary plot plus a pandas.DataFrame of feature importances.

from endgame.explain import explain

# Works with tree models, linear models, neural nets, or black-box estimators
report = explain(model, X_test)

# report.importances  — DataFrame, one row per feature
# report.plot()       — displays a bar chart of global importances

Auto-detection rules:

Model type	Method selected
Tree-based (GBDT, RF, etc.)	TreeSHAP
Linear / GLM	LinearSHAP
Neural network (PyTorch)	DeepSHAP
Any other estimator	KernelSHAP (slower, model-agnostic)

Pass method='lime' or method='shap' to override auto-detection.

SHAP Explanations¶

SHAPExplainer wraps the four SHAP backends behind a single class. It selects the correct backend at fit time based on the model type.

from endgame.explain import SHAPExplainer

explainer = SHAPExplainer(model)
explainer.fit(X_train)           # background dataset for KernelSHAP / DeepSHAP

# Global explanation — mean absolute SHAP value per feature
global_imp = explainer.global_importance(X_test)
explainer.plot_bar(X_test)       # bar chart of global importances
explainer.plot_beeswarm(X_test)  # beeswarm (violin-style) summary plot

# Local explanation — SHAP values for a single prediction
shap_values = explainer.explain(X_test)   # shape: (n_samples, n_features)
explainer.plot_waterfall(X_test[0])       # waterfall for the first sample
explainer.plot_force(X_test[0])           # force plot

Interaction values¶

# Pairwise SHAP interaction values (TreeSHAP only)
interactions = explainer.interaction_values(X_test)  # (n, n_features, n_features)
explainer.plot_interaction_heatmap(X_test)

LIME Explanations¶

LIMEExplainer fits a local linear model around each prediction using perturbed samples. It is model-agnostic and works with any predict_proba or predict method.

from endgame.explain import LIMEExplainer

lime = LIMEExplainer(model, mode='classification')  # 'classification' | 'regression'
lime.fit(X_train)

# Explain one instance
explanation = lime.explain(X_test[0], num_features=10)
explanation.show_in_notebook()       # interactive HTML widget
print(explanation.as_dataframe())    # feature weights as DataFrame

# Explain a batch and aggregate
batch_exp = lime.explain_batch(X_test[:50], num_features=10)
lime.plot_summary(batch_exp)

Partial Dependence¶

PartialDependence computes marginal effect of one or two features while averaging over all other features. Use this to understand the shape of a feature’s relationship with the target.

from endgame.explain import PartialDependence

pdp = PartialDependence(model)
pdp.fit(X_train)

# 1D partial dependence
pdp.plot(X_test, feature='age')

# 2D interaction surface (two features simultaneously)
pdp.plot_2d(X_test, features=['age', 'income'])

# Individual Conditional Expectation (ICE) curves
pdp.plot_ice(X_test, feature='age', n_samples=50, center=True)

# Return raw values without plotting
grid, avg, individual = pdp.compute(X_test, feature='age')

Feature Interactions¶

FeatureInteraction measures pairwise and higher-order interactions using Friedman’s H-statistic. High H-statistic values indicate that the joint effect of two features cannot be decomposed into separate individual effects.

from endgame.explain import FeatureInteraction

fi = FeatureInteraction(model)
fi.fit(X_train)

# Overall interaction strength for every feature (one number per feature)
h_overall = fi.overall_strength(X_test)

# Pairwise H-statistic matrix
h_matrix = fi.pairwise(X_test)          # DataFrame, (n_features, n_features)
fi.plot_heatmap(X_test)                  # colour-coded interaction matrix

# Top-k interacting pairs
top_pairs = fi.top_pairs(X_test, k=5)
print(top_pairs)
# feature_a  feature_b  h_statistic
# age        income       0.34
# ...

Counterfactual Explanations¶

CounterfactualExplainer generates the smallest-change perturbation of an input that flips the model prediction to a desired outcome class. The implementation is based on DiCE (Diverse Counterfactual Explanations).

from endgame.explain import CounterfactualExplainer

cf = CounterfactualExplainer(
    model,
    feature_names=X_train.columns.tolist(),
    categorical_features=['gender', 'education'],
)
cf.fit(X_train)

# Generate counterfactuals for one instance
counterfactuals = cf.explain(
    X_test[0],
    desired_class=1,         # the outcome we want
    n_counterfactuals=5,     # number of diverse alternatives
    proximity_weight=0.5,    # penalise large changes
    diversity_weight=1.0,    # encourage variety
)
print(counterfactuals.as_dataframe())

# Feature ranges can be constrained to realistic values
cf_constrained = CounterfactualExplainer(
    model,
    feature_names=X_train.columns.tolist(),
    feature_ranges={'age': (18, 80), 'income': (0, 500_000)},
    immutable_features=['gender'],
)
cf_constrained.fit(X_train)
result = cf_constrained.explain(X_test[0], desired_class=1)

Comparing Multiple Explanation Methods¶

Use explain() with method='all' to run every applicable method and return a side-by-side comparison:

from endgame.explain import explain

report = explain(model, X_test, method='all', n_samples=100)
report.compare()    # prints agreement metrics across methods
report.plot_all()   # grid of summary plots

Saving Explanation Reports¶

report = explain(model, X_test)
report.save_html('explanation_report.html')   # standalone HTML with plots
report.save_json('explanation_report.json')   # feature importances as JSON

API Reference¶

Full parameter documentation for explain, SHAPExplainer, LIMEExplainer, PartialDependence, FeatureInteraction, and CounterfactualExplainer is available in the auto-generated API reference at docs/api/explain.rst or by calling help(ClassName) at the Python prompt.