Chapter 2: Credit Scoring Models

Overview of Credit Scoring Models

Credit scoring models transform borrower data into quantitative risk assessments. This chapter explores the evolution from traditional scorecard methods to modern AI/ML approaches.

Traditional Scorecard Models

Logistic Regression Scorecards

The foundation of traditional credit scoring, logistic regression models estimate the probability of default based on weighted characteristics:

# Traditional Scorecard Example
Score = Base Points +
        (Payment History × 35) +
        (Credit Utilization × 30) +
        (Credit History Length × 15) +
        (New Credit × 10) +
        (Credit Mix × 10)

# Typical Implementation:
- 10-30 features
- Linear relationships
- Manual segmentation
- Periodic recalibration
                

Advantages:

• Highly interpretable and explainable
• Regulatory approval and compliance
• Stable and consistent performance
• Low computational requirements

Limitations:

• Assumes linear relationships
• Limited feature interactions
• Manual feature engineering required
• Cannot adapt to complex patterns

Credit Bureau Scores

FICO Score 8 (Most Common)

Developed in 2009, still widely used by lenders. Focuses on:

• Multiple late payments weigh heavily
• High credit utilization penalized
• Isolated late payments less damaging
• Small collection amounts ignored

VantageScore 3.0/4.0

Competitive model from Experian, Equifax, TransUnion:

• Can score thin-file consumers
• Shorter credit history required (1 month vs. 6 months)
• Trending data incorporated
• Machine learning enhancements in 4.0

Modern Machine Learning Models

Gradient Boosting Machines

XGBoost and LightGBM have become industry standards for credit scoring due to exceptional performance:

import xgboost as xgb
from sklearn.model_selection import train_test_split

# XGBoost Credit Scoring Model
model = xgb.XGBClassifier(
    max_depth=6,
    learning_rate=0.1,
    n_estimators=100,
    objective='binary:logistic',
    subsample=0.8,
    colsample_bytree=0.8,
    scale_pos_weight=10  # Handle class imbalance
)

# Train on historical loan performance
X_train, X_test, y_train, y_test = train_test_split(
    features, defaults, test_size=0.3
)

model.fit(X_train, y_train)

# Predict default probability
default_prob = model.predict_proba(X_test)[:, 1]
credit_score = 300 + (850 - 300) * (1 - default_prob)
                

Key Benefits:

• Handles 500+ features automatically
• Captures non-linear relationships
• Built-in feature importance
• Resistant to overfitting
• Excellent predictive accuracy

Neural Networks

Deep learning models for complex pattern recognition:

import tensorflow as tf
from tensorflow import keras

# Deep Neural Network for Credit Scoring
model = keras.Sequential([
    keras.layers.Dense(256, activation='relu', input_dim=n_features),
    keras.layers.Dropout(0.3),
    keras.layers.Dense(128, activation='relu'),
    keras.layers.Dropout(0.2),
    keras.layers.Dense(64, activation='relu'),
    keras.layers.Dense(1, activation='sigmoid')
])

model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['AUC', 'precision', 'recall']
)

# Train with early stopping
model.fit(
    X_train, y_train,
    validation_split=0.2,
    epochs=50,
    batch_size=256,
    callbacks=[keras.callbacks.EarlyStopping(patience=5)]
)
                

Applications:

• Alternative data integration (text, transaction patterns)
• Embedding categorical variables
• Time-series credit behavior modeling
• Multi-task learning (score + explain + recommend)

Ensemble Methods

Combining multiple models for superior performance:

Model Comparison

Specialized Models

Thin-File Scoring

Models designed for consumers with limited credit history using alternative data:

• Utility payment history (12-24 months)
• Rent payment records
• Telecom and mobile phone payments
• Bank account transaction patterns
• Education and employment verification

SMB/Commercial Scoring

Business credit assessment incorporating:

• Business financial statements
• Trade credit and payment patterns
• Business owner personal credit
• Industry risk factors
• Revenue trends and cash flow

Behavioral Scoring

Ongoing risk assessment for existing customers:

• Account usage patterns
• Payment behavior changes
• Balance trends
• New credit-seeking activity
• Credit line utilization dynamics

Model Development Lifecycle

1. Data Collection: Gather 2-3 years of historical applications with outcomes
2. Data Preparation: Clean, normalize, handle missing values
3. Feature Engineering: Create predictive variables from raw data
4. Model Training: Develop multiple candidate models
5. Validation: Test on hold-out data, assess performance
6. Calibration: Ensure predicted probabilities match actual default rates
7. Approval: Model risk management review and regulatory approval
8. Deployment: Production implementation with monitoring
9. Monitoring: Track performance, detect drift
10. Retraining: Quarterly or annual model updates

Best Practices

In the next chapter, we'll explore the data sources and feature engineering techniques that power these models.