Chapter 4

Impact Analysis

Impact analysis answers the critical question: "What will break if I make this change?" By leveraging data lineage, teams can proactively identify downstream dependencies and prevent catastrophic failures before they happen.

What is Impact Analysis?

Impact analysis uses lineage graphs to trace all downstream consumers of a data asset. When you plan to modify a table, column, or transformation, impact analysis shows you every report, dashboard, ML model, and data pipeline that depends on it.

            Core Principle: Better to discover breaking changes during planning than during a production outage at 3am.
        

Types of Changes to Analyze

1. Schema Changes

Column Rename: Updates all downstream references
Column Drop: Identifies what will break
Data Type Change: Finds incompatible usage
Table Rename/Move: Updates all consumers

2. Business Logic Changes

Modifying SQL transformation logic
Changing aggregation methods
Updating calculation formulas
Altering filtering conditions

3. Infrastructure Changes

Data warehouse migration
Database engine upgrades
API version changes
Storage format conversions

Real-World Scenario

Situation: Your team wants to rename customer_id to user_id for consistency.

Without Impact Analysis: Deploy the change → 47 dashboards break → 12 ETL jobs fail → ML model predictions stop → executive dashboard is down → emergency rollback → lost productivity and trust.

With Impact Analysis: Run analysis → identify 47 downstream dependencies → create migration plan → update consumers in stages → deploy safely → zero downtime.

Performing Impact Analysis

Step 1: Identify the Change

Target: warehouse.customers.email_address
Change Type: Column rename
New Name: contact_email
Reason: Standardization

Step 2: Query Lineage Graph

SELECT DISTINCT
    downstream_table,
    downstream_column,
    transformation_type,
    owner
FROM lineage_metadata
WHERE upstream_table = 'warehouse.customers'
  AND upstream_column = 'email_address'
ORDER BY depth DESC;

Step 3: Analyze Results

Found 23 downstream dependencies:

DIRECT DEPENDENCIES (Depth 1):
  ✓ analytics.customer_summary.email
  ✓ reports.marketing_list.contact_email
  ✓ ml_features.customer_attributes.email_domain

INDIRECT DEPENDENCIES (Depth 2):
  ⚠ dashboards.executive_overview (via customer_summary)
  ⚠ ml_models.churn_predictor (via customer_attributes)

CRITICAL DEPENDENCIES:
  ⛔ real_time_api.customer_lookup (SLA: <100ms)

Step 4: Risk Assessment

High Risk: Real-time APIs, executive dashboards
Medium Risk: Daily reports, ML models
Low Risk: Archived datasets, exploratory notebooks

Step 5: Migration Planning

Add new column while keeping old (deprecation period)
Update non-critical consumers first
Update critical systems during maintenance window
Remove old column after validation period

Automated Impact Analysis

Example: CI/CD Integration

# Pre-deployment impact check
name: Impact Analysis
on: pull_request

jobs:
  analyze-impact:
    runs-on: ubuntu-latest
    steps:
      - name: Extract schema changes
        run: |
          git diff main --name-only | grep "\.sql$" > changed_files.txt

      - name: Run impact analysis
        run: |
          python scripts/analyze_impact.py \
            --changes changed_files.txt \
            --output impact_report.md

      - name: Comment on PR
        if: impact.high_risk_count > 0
        run: |
          gh pr comment ${{ github.event.pull_request.number }} \
            --body-file impact_report.md

Impact Analysis Patterns

Forward Impact (Downstream)

Most common: "What depends on this?"

Traverse lineage graph forward
Find all consumers
Assess criticality

Backward Impact (Upstream)

"What does this depend on?"

Useful for deprecation decisions
Understanding data freshness requirements
Identifying single points of failure

Lateral Impact (Same Level)

"What else uses the same source?"

Find related datasets
Coordinate parallel changes
Identify consolidation opportunities

Impact Metrics

Quantify the blast radius of changes:

Affected Assets: Count of downstream dependencies
Affected Users: Number of people/teams impacted
Criticality Score: Weighted importance of affected systems
Depth: How many hops downstream
Estimated Effort: Hours to update all consumers

Advanced Techniques

What-If Analysis

Model hypothetical changes before making them:

WHAT IF we delete staging.temp_customer_data?
  → orphans: analytics.customer_360 (broken)
  → requires: new source for customer_360
  → estimated migration: 16 hours

Blast Radius Visualization

Visual representation of impact scope:

Concentric circles showing impact depth
Color-coded by risk level
Interactive drill-down

Time-Based Impact

Consider when changes take effect:

Batch jobs: impact at next run
Real-time: immediate impact
Cached data: delayed impact

Best Practices

Run Impact Analysis Early: During design, not just before deployment
Automate Where Possible: Integrate with CI/CD pipelines
Set Thresholds: Define acceptable impact levels
Require Sign-Off: High-impact changes need stakeholder approval
Document Decisions: Record why changes were deemed acceptable
Plan Rollback: Always have a way to undo changes
Communicate Proactively: Notify affected teams in advance

            Key Takeaway: Impact analysis transforms data lineage from documentation into a proactive risk management tool. It's the difference between reactive firefighting and planned, coordinated changes.
        