Explanation Types and Frameworks

Understanding Explanation Taxonomies

Explainable AI encompasses a diverse ecosystem of methods, techniques, and approaches. To navigate this complexity, we need robust taxonomies that categorize explanations along multiple dimensions. This chapter provides a comprehensive framework for understanding different types of explanations and when to apply them.

This chapter organizes explanation methods along several key axes: scope (local vs. global), model dependency (model-specific vs. model-agnostic), explanation format (feature importance, rules, examples, counterfactuals), and temporal characteristics (static vs. interactive). Understanding these dimensions enables practitioners to select appropriate methods for their specific contexts.

Local vs. Global Explanations

The most fundamental distinction in XAI is between local and global explanations. This dichotomy reflects different questions we might ask about model behavior.

Local Explanations: Understanding Individual Predictions

Local explanations answer the question: "Why did the model make this specific prediction for this particular input?" They focus on a single instance and explain the model's behavior at that point in the feature space.

Local explanations are crucial for:

• Individual decision-making: Helping people understand outcomes that affect them personally
• Regulatory compliance: Satisfying requirements like GDPR's right to explanation
• Case-by-case review: Enabling human oversight of automated decisions
• Error analysis: Understanding why the model failed on specific examples
• Building trust: Demonstrating that individual decisions are reasonable

Global Explanations: Understanding Model Behavior

Global explanations answer: "How does the model behave overall?" They characterize the model's decision patterns across the entire input space or significant portions thereof.

Global explanations support:

• Model validation: Verifying that the model learned sensible patterns
• Bias detection: Identifying unwanted dependencies on protected attributes
• Scientific discovery: Learning new insights from data
• Model comparison: Understanding how different models make different trade-offs
• Regulatory compliance: Demonstrating overall fairness and non-discrimination

The Local-Global Connection

While local and global explanations serve different purposes, they are interconnected. Aggregating many local explanations can provide global insights, and global patterns help contextualize local decisions. Some methods, like SHAP, provide both local explanations for individual predictions and global explanations through aggregation.

Model-Agnostic vs. Model-Specific Methods

Another crucial dimension distinguishes methods by their relationship to the underlying model architecture.

Model-Agnostic Approaches

Model-agnostic methods treat the AI model as a black box. They work by observing input-output relationships without needing to understand internal model structure. This "outside looking in" approach offers significant advantages:

• Universal applicability: The same explanation method works for any model type
• Flexibility: Can compare explanations across different model architectures
• Future-proofing: Methods remain useful as new model types emerge
• Simplicity: No need to understand complex model internals

Popular model-agnostic methods include LIME (Local Interpretable Model-agnostic Explanations), SHAP (when used in model-agnostic mode), permutation feature importance, and partial dependence plots.

Model-Specific Approaches

Model-specific methods leverage knowledge of the model's internal structure to provide deeper insights. Different model types afford different explanation opportunities:

Model Type	Specific Explanation Methods	Advantages
Linear Models	Coefficient interpretation, standardized coefficients	Direct, mathematically exact
Decision Trees	Path tracing, rule extraction	Intuitive if-then logic
Neural Networks	Attention weights, saliency maps, layer-wise relevance propagation (LRP)	Visual, highlights important regions
Tree Ensembles	TreeSHAP, feature interactions	Fast, captures nonlinear effects

Model-specific methods can be more efficient and provide richer insights, but at the cost of requiring different implementations for different model types.

Explanation Output Formats

Explanations can be presented in various formats, each suited to different contexts and audiences. This book divides them into five primary format categories. The WIA-AI-009 specification (Phase 1) standardizes the data formats for local, global, and counterfactual explanations among them.

1. Feature Importance Explanations

The most common explanation format assigns importance scores to input features, indicating their contribution to the prediction. These can be presented as ranked lists, bar charts, or numerical attributions.

// Attribution vector format (WIA-AI-009 Phase 1 AttributionVector)
{
  "explanation_type": "local",
  "method": "shap",
  "explanation": {
    "attributions": [
      { "feature_name": "credit_score", "attribution_value": 0.42,
        "direction": "positive", "rank": 1 },
      { "feature_name": "debt_ratio", "attribution_value": -0.35,
        "direction": "negative", "rank": 2 }
      // ...
    ]
  }
}

2. Rule-Based Explanations

Rule-based explanations express the model's logic as human-readable if-then statements. While most naturally associated with decision trees, rules can be extracted from any model.

3. Example-Based Explanations

Example-based methods explain predictions by reference to similar training examples or prototypes. "This tumor is classified as malignant because it closely resembles these five training examples, all of which were malignant."

Techniques include:

• Influential instances: Training examples that most affected the model's behavior
• Prototypes: Representative examples of each class
• Criticisms: Examples the model handles poorly, revealing limitations
• Nearest neighbors: Most similar instances in the training data

4. Counterfactual Explanations

Counterfactuals answer "what if" questions: "What would need to change for the model to make a different prediction?" They are inherently actionable, showing users how to achieve desired outcomes.

Effective counterfactuals should be:

• Minimal: Change as few features as possible
• Actionable: Propose changes users can actually make
• Realistic: Stay within plausible feature ranges
• Sparse: Avoid unnecessary complexity

5. Visual Explanations

For domains like computer vision, visual explanations highlight which parts of an image influenced the prediction. Techniques include saliency maps, attention heatmaps, and concept activation vectors.

Intrinsic vs. Post-hoc Interpretability

This dimension distinguishes between models that are inherently interpretable and explanation methods applied after training.

Intrinsically Interpretable Models

Some model architectures are designed to be interpretable from the ground up. Their predictions can be understood by examining their structure directly:

• Linear/Logistic Regression: Each coefficient shows a feature's effect
• Decision Trees: Predictions follow explicit decision paths
• Rule Lists: Ordered if-then rules evaluated sequentially
• Generalized Additive Models (GAMs): Sum of univariate functions
• Attention-based Neural Nets: Built-in attention weights show focus areas

Post-hoc Explanation Methods

Post-hoc methods are applied to trained models to explain their behavior. They work with any model, including black boxes, but their explanations are necessarily approximations.

The key trade-off: intrinsic interpretability guarantees that explanations accurately reflect the model's true reasoning, but may limit model expressiveness. Post-hoc methods work with arbitrarily complex models but their explanations are approximations that might not fully capture model behavior.

Static vs. Interactive Explanations

Explanations can be presented as static summaries or as interactive tools that users can explore.

Static Explanations

Static explanations provide a fixed summary of model behavior. They're suitable for formal documentation, regulatory submissions, and scenarios where users need consistent, reproducible explanations.

Interactive Explanations

Interactive tools let users explore model behavior dynamically—tweaking inputs to see how predictions change, drilling down into specific features, or filtering by subpopulations. This supports deeper understanding and discovery of non-obvious patterns.

This book recommends interactive explanations for expert users (data scientists, auditors) while recommending static summaries for end-users to avoid overwhelming them with complexity.

Temporal Aspects: Explaining Model Evolution

Machine learning models change over time through retraining, online learning, or concept drift. Temporal explanations address how and why model behavior evolves.

Version-to-Version Comparison

When deploying a new model version, stakeholders want to understand what changed. Did the new model learn different patterns? Are there subpopulations where predictions shifted significantly?

Temporal Feature Importance

In online learning scenarios, feature importance may shift as data distributions change. Tracking these shifts helps detect concept drift and data quality issues.

Audience-Specific Explanation Design

Effective explanations are tailored to their audience. The request format of the WIA-AI-009 specification (Phase 3) specifies four target audiences—technical, business, enduser, and regulatory—and this book adds domain experts to describe five explanation profiles.

Technical Audience Profile

Target: Data scientists, ML engineers, researchers
Format: Detailed feature attributions, statistical measures, mathematical formulations
Visualization: Technical plots (partial dependence, SHAP summary plots, confusion matrices)
Depth: Full detail including confidence intervals, p-values, algorithmic parameters

Domain Expert Profile

Target: Doctors, financial analysts, subject matter experts
Format: Domain-relevant features, comparisons to expert reasoning
Visualization: Domain-specific representations (medical images with highlights, financial charts)
Depth: Emphasis on validating against domain knowledge

Business Stakeholder Profile

Target: Executives, product managers, decision-makers
Format: High-level summaries, business metrics, risk assessments
Visualization: Executive dashboards, ROI analyses
Depth: Strategic implications, not technical details

End-User Profile

Target: People affected by AI decisions
Format: Plain language, actionable recommendations
Visualization: Simple charts, icons, natural language
Depth: Focus on "what" and "how to respond," not "why" in technical terms

Regulatory/Audit Profile

Target: Auditors, regulators, compliance officers
Format: Comprehensive documentation, audit trails, fairness metrics
Visualization: Compliance dashboards, statistical evidence of non-discrimination
Depth: Sufficient detail to verify legal/ethical compliance

Multi-Level Explanation Hierarchies

Complex systems often require explanations at multiple levels of abstraction. This book recommends the following hierarchical approach. The granularity setting (granularity: high, medium, low) in the specification's request format can express these differences in level.

1. Level 1: Prediction Summary
   "Your loan was denied" or "Malignant tumor detected with 87% confidence"
2. Level 2: Primary Factors
   "Main reasons: high debt ratio, short employment history"
3. Level 3: Detailed Attribution
   Numerical importance scores, counterfactuals, feature interactions
4. Level 4: Technical Deep Dive
   SHAP values, confidence intervals, similar training examples, model internals

Users can drill down through levels as needed. End-users typically need only levels 1-2, while auditors might require all four levels.

Composite Explanations

Real-world applications often benefit from combining multiple explanation types. A comprehensive explanation might include:

• Feature importance (what mattered most)
• A counterfactual (how to change the outcome)
• Similar examples (how this compares to past cases)
• Confidence assessment (how certain is the prediction)
• Temporal context (how this prediction compares to previous model versions)

The WIA-AI-009 specification's response format (XAI-RESPONSE-v1) carries explanations from multiple methods in its explanations array and lets requesters receive counterfactuals as well via include_counterfactuals, so users gain comprehensive understanding without needing multiple separate queries.

Contextual Explanation Selection

Not all explanations are appropriate for all situations. The following table summarizes criteria for selecting explanation methods based on context. For method selection by model type, also consult the method selection guidance in Phase 2 of the specification.

Context	Recommended Approach	Rationale
Individual decision appeals	Local + Counterfactual	Actionable guidance for affected individuals
Model validation	Global + Feature interactions	Comprehensive understanding of model behavior
Bias auditing	Subgroup analysis + Fairness metrics	Detect disparate impact across demographics
Real-time decision support	Fast local methods (TreeSHAP for tree models, attention, precomputed and cached explanations)	Low latency requirements
Scientific discovery	Global + Domain visualization	Uncover novel insights from data