SHAP: Shapley Additive Explanations

Introduction to SHAP

SHAP (SHapley Additive exPlanations) represents one of the most theoretically grounded and widely adopted approaches to explainable AI. Developed by Scott Lundberg and Su-In Lee in 2017, SHAP builds on game theory concepts to provide a unified framework for interpreting model predictions. Its mathematical rigor and desirable theoretical properties make it a cornerstone of the WIA-AI-009 standard.

At its core, SHAP answers a deceptively simple question: How much did each feature contribute to moving the model's prediction away from the average prediction? The answer provides both local explanations for individual predictions and, through aggregation, global insights into model behavior. This dual capability, combined with provable guarantees about explanation quality, makes SHAP exceptionally valuable for high-stakes applications.

Game Theory Foundation: Shapley Values

To understand SHAP, we must first understand Shapley values, a concept from cooperative game theory introduced by Lloyd Shapley in 1951 (for which he later won the Nobel Prize in Economics). The fundamental problem Shapley addressed was: In a cooperative game where players work together to achieve a total payoff, how should that payoff be fairly distributed among the players?

The Cooperative Game Analogy

Consider a simple example: Three friends—Alice, Bob, and Carol—start a company together. After a year, the company makes $300,000 in profit. How should this be divided among the three founders?

A naive approach might split it equally: $100,000 each. But what if we know that:

• Alice alone could generate $80,000
• Bob alone could generate $60,000
• Carol alone could generate $90,000
• Alice + Bob together could generate $180,000
• Alice + Carol together could generate $200,000
• Bob + Carol together could generate $170,000
• All three together generate $300,000

The Shapley value provides a principled answer: Calculate each person's average marginal contribution across all possible orderings of team formation. If we computed this (we'll see how shortly), we might find that Alice deserves $105,000, Bob $92,000, and Carol $103,000—reflecting each person's actual contribution to the team's success.

From Business to Machine Learning

SHAP applies this same logic to machine learning:

• Players = Features in your dataset
• Payoff = Model's prediction
• Coalition = Subset of features used to make a prediction
• Shapley value = Each feature's fair contribution to the prediction

Just as we asked "How much did Alice contribute to the company's success?" SHAP asks "How much did the credit score feature contribute to this loan approval prediction?"

Mathematical Definition of SHAP Values

The SHAP value for feature i is defined as:

Let's break down what this formula means:

• S: A subset of features (a "coalition") that doesn't include feature i
• f(S ∪ {i}) - f(S): The marginal contribution of adding feature i to coalition S
• |S|! × (M - |S| - 1)! / M!: A weighting factor ensuring all orderings are considered equally
• Sum over all S: Average the marginal contributions across all possible coalitions

Intuitive Interpretation

Imagine you're trying to predict loan default risk. You have 10 features. To compute the SHAP value for "credit_score":

1. Consider every possible subset of the other 9 features
2. For each subset, make predictions with and without credit_score
3. The difference is credit_score's marginal contribution for that subset
4. Average all these marginal contributions (weighted appropriately)
5. This average is credit_score's SHAP value

With 10 features, there are 2⁹ = 512 subsets to consider. With 100 features, there are 2⁹⁹ subsets— computationally infeasible to enumerate. This is why practical SHAP implementations use approximations and optimizations, which we'll explore shortly.

Desirable Properties of SHAP

SHAP values satisfy four crucial axioms that make them theoretically attractive:

1. Local Accuracy (Additive Property)

The prediction equals the base value (average prediction) plus the sum of all SHAP values:

This means SHAP values fully account for the prediction. There's no unexplained residual. If the model predicts 0.8 and the base value is 0.5, the SHAP values must sum to 0.3.

2. Missingness

If a feature is missing (or has no impact on the model), its SHAP value is zero. This seems obvious but not all explanation methods satisfy this property.

3. Consistency (Monotonicity)

If changing a model increases a feature's marginal contribution everywhere, that feature's SHAP value cannot decrease. This ensures that explanations change consistently with model changes.

4. Symmetry

If two features contribute equally to all predictions, they receive equal SHAP values. This prevents arbitrary inequality in attribution.

SHAP Variants and Computational Approaches

Computing exact SHAP values is exponentially expensive. The original SHAP paper introduced several approximation methods optimized for different model types:

TreeSHAP

For tree-based models (decision trees, random forests, XGBoost, LightGBM), TreeSHAP computes exact SHAP values in polynomial time by exploiting the tree structure. Instead of enumerating all feature subsets, it traces prediction paths through the trees.

TreeSHAP is remarkably efficient—often only 2-3x slower than making predictions—making it practical for production use. This efficiency, combined with the popularity of tree-based models, makes TreeSHAP one of the most widely deployed XAI methods.

// WIA-AI-009 TreeSHAP implementation example
import { TreeSHAPExplainer } from 'wia-ai-009';

const model = loadXGBoostModel('loan_model.json');
const explainer = new TreeSHAPExplainer(model);

const shapValues = explainer.explain({
  credit_score: 720,
  debt_ratio: 0.35,
  employment_years: 8,
  loan_amount: 250000
});

console.log(shapValues);
// {
//   credit_score: -0.22,
//   debt_ratio: 0.45,
//   employment_years: -0.18,
//   loan_amount: 0.12,
//   base_value: 0.15,
//   prediction: 0.32
// }

KernelSHAP

KernelSHAP is model-agnostic, working with any model type. It approximates SHAP values by:

1. Sampling feature coalitions (random subsets of features)
2. Making predictions with those coalitions
3. Fitting a weighted linear model to estimate each feature's contribution
4. The linear model's coefficients approximate SHAP values

KernelSHAP trades accuracy for flexibility. With sufficient samples (typically 1,000-10,000), it provides good approximations even for complex neural networks.

LinearSHAP

For linear models, SHAP values have a closed-form solution. If your model is:

Then the SHAP value for feature i is simply:

This means the SHAP value is the coefficient times how much the feature value differs from its average. LinearSHAP is instantaneous and exact.

DeepSHAP

DeepSHAP approximates SHAP values for deep neural networks by backpropagating contribution scores through the network. It's faster than KernelSHAP for neural nets but makes approximations that may not satisfy all SHAP axioms perfectly.

GradientSHAP

For differentiable models, GradientSHAP uses gradients to estimate SHAP values. It randomly samples points along the path from a baseline to the input and averages the gradients, similar to Integrated Gradients.

Visualizing SHAP Values

SHAP provides several powerful visualization techniques that make complex model behavior accessible:

Force Plots (Local Explanations)

Force plots show how each feature pushes the prediction higher or lower from the base value. Red features increase the prediction, blue features decrease it. The width indicates magnitude.

Summary Plots (Global Feature Importance)

Summary plots aggregate SHAP values across many instances to show global feature importance. They display:

• Features ranked by average absolute SHAP value (importance)
• Distribution of SHAP values (spread shows how much impact varies)
• Color indicates feature value (high/low)

A summary plot might reveal that credit_score is the most important feature overall, and high credit scores consistently decrease default risk while low scores increase it.

Dependence Plots

Dependence plots show how SHAP values for a feature vary with that feature's value, revealing the relationship between feature value and prediction impact.

For example, plotting credit_score on the x-axis and its SHAP value on the y-axis might show a nonlinear relationship: SHAP values drop sharply as credit score increases from 600 to 700, then flatten above 750. This reveals that improving a poor credit score has more impact than further improving a good score.

Interaction Values

SHAP can be extended to capture feature interactions—how the combined effect of two features differs from their individual effects. This helps identify synergies and dependencies between features.

Implementing SHAP in Production

Deploying SHAP in production systems requires attention to performance, consistency, and user experience:

Performance Optimization

Use the Right Variant: TreeSHAP for tree models, LinearSHAP for linear models, KernelSHAP only when necessary.

Batch Processing: If real-time explanations aren't required, compute SHAP values in batch overnight and cache them.

Approximation Tuning: For KernelSHAP, balance sample count against latency requirements. Start with 100 samples for debugging, increase to 1,000+ for production.

Background Data Selection: SHAP requires a background dataset representing "typical" data. Use a representative sample of training data (100-1,000 instances typically suffice).

Consistency Across Requests

Randomized SHAP variants (KernelSHAP) can produce slightly different explanations on repeated requests for the same input. To ensure consistency:

• Set random seeds for reproducibility
• Cache explanations for identical inputs
• Use deterministic variants (TreeSHAP, LinearSHAP) when possible

Integration with WIA-AI-009 Format

// WIA-AI-009 standardized SHAP output
{
  "standard": "WIA-AI-009",
  "version": "1.0",
  "explanation_type": "shap",
  "timestamp": "2025-01-20T14:30:00Z",
  "model_info": {
    "model_id": "loan_model_v2.3",
    "model_type": "xgboost",
    "shap_variant": "TreeSHAP"
  },
  "prediction": {
    "value": 0.48,
    "class": "default_risk",
    "confidence": 0.87
  },
  "explanation": {
    "base_value": 0.15,
    "shap_values": {
      "credit_score": -0.22,
      "debt_ratio": 0.45,
      "employment_years": -0.18,
      "recent_inquiries": 0.28,
      "loan_amount": 0.12,
      "age": -0.03
    },
    "feature_values": {
      "credit_score": 720,
      "debt_ratio": 0.52,
      "employment_years": 8,
      "recent_inquiries": 5,
      "loan_amount": 250000,
      "age": 35
    }
  },
  "metadata": {
    "computation_time_ms": 145,
    "background_data_size": 500,
    "shap_sum_check": 0.42
  }
}

SHAP for Different Model Types

The WIA-AI-009 standard provides guidance for applying SHAP across diverse model architectures:

Model Type	Recommended SHAP Variant	Typical Latency	Exactness
Linear/Logistic Regression	LinearSHAP	< 1ms	Exact
Decision Trees	TreeSHAP	1-10ms	Exact
Random Forest, XGBoost	TreeSHAP	10-100ms	Exact
Neural Networks	DeepSHAP or KernelSHAP	100-5000ms	Approximate
SVM, Other	KernelSHAP	500-10000ms	Approximate

Limitations and Considerations

While SHAP is powerful, it's important to understand its limitations:

Computational Cost

Even with optimizations, SHAP can be expensive for high-dimensional data or complex models. Real-time explanations may be impractical without careful engineering.

Correlation vs. Causation

SHAP values reflect statistical associations in the model, not causal relationships. A high SHAP value means a feature is correlated with the prediction, but changing that feature in the real world might not change the outcome due to confounding variables.

Baseline Dependence

SHAP values depend on the choice of baseline (background dataset). Different baselines can yield different values. The WIA-AI-009 standard recommends using a representative sample of training data as the baseline.

Interpretation Challenges

For models with hundreds of features, SHAP produces hundreds of attribution values per prediction. Presenting all of them overwhelms users. In practice, show only the top 5-10 features by absolute SHAP value.

Correlated Features

When features are highly correlated, SHAP may distribute importance between them in ways that seem arbitrary. The total importance is correct, but individual attributions may vary due to multicollinearity.

Advanced SHAP Techniques

Beyond basic feature attribution, SHAP enables sophisticated analyses:

Conditional Expectations

Instead of marginalizing over all features, condition on realistic feature combinations. This prevents SHAP from considering impossible feature combinations (e.g., "age=20 AND employment_years=30").

Clustered SHAP

Group similar instances and show aggregate SHAP patterns for each cluster. This reveals that the model behaves differently for different subpopulations.

SHAP-based Model Selection

Compare SHAP values across different models to understand not just which performs better, but how their reasoning differs. Model A might rely heavily on feature X while Model B favors feature Y—informing which model to deploy based on domain appropriateness.

Temporal SHAP

Track how SHAP values change over time as models are retrained. Sudden shifts in feature importance can signal concept drift or data quality issues.