← Back to Contents

Chapter 02: Privacy-Preserving Techniques

Master cryptographic and statistical methods for protecting data in federated learning

The Privacy Challenge in Federated Learning

While federated learning prevents raw data from leaving client devices, model updates themselves can leak sensitive information. Researchers have demonstrated that gradient information can be used to reconstruct training samples, infer membership, and extract private attributes. This chapter explores techniques to strengthen privacy guarantees in federated learning systems.

Differential Privacy

Fundamental Concept

Differential Privacy (DP) provides a mathematical framework for quantifying privacy loss. A randomized algorithm M satisfies (ε, δ)-differential privacy if for any two datasets D and D' differing in a single record, and for all possible outputs S:

Pr[M(D) ∈ S] ≤ exp(ε) × Pr[M(D') ∈ S] + δ

Where:

• ε (epsilon): Privacy budget - smaller values mean stronger privacy
• δ (delta): Probability of privacy breach - typically very small (e.g., 10⁻⁵)

Privacy Budget Guidelines

Epsilon Range	Privacy Level	Typical Use Cases
ε < 1	Very Strong	Medical records, financial data
ε = 1-3	Strong	Personal preferences, location history
ε = 3-6	Moderate	Aggregated statistics, demographic info
ε > 6	Weak	Public or semi-public data

Differential Privacy in Federated Learning

Two main approaches exist for applying differential privacy to federated learning:

1. Local Differential Privacy (LDP)

Each client adds noise to their model updates before sending to the server. This provides the strongest privacy guarantee as even the server cannot see clean updates.

def add_local_noise(gradient, epsilon, sensitivity):
    """
    Add Laplace noise for local differential privacy

    Args:
        gradient: Model gradient computed on local data
        epsilon: Privacy budget
        sensitivity: L2 sensitivity of gradient

    Returns:
        Noisy gradient satisfying epsilon-DP
    """
    scale = sensitivity / epsilon
    noise = np.random.laplace(0, scale, gradient.shape)
    noisy_gradient = gradient + noise

    # Clip gradient to bound sensitivity
    clip_norm = 1.0
    gradient_norm = np.linalg.norm(noisy_gradient)
    if gradient_norm > clip_norm:
        noisy_gradient = noisy_gradient * (clip_norm / gradient_norm)

    return noisy_gradient

2. Central Differential Privacy (CDP)

The server adds noise after aggregating updates. Provides better utility (accuracy) but requires trusting the server to see clean aggregated updates.

def aggregate_with_central_dp(client_updates, epsilon, delta, num_clients):
    """
    Aggregate client updates with central differential privacy

    Uses Gaussian mechanism for (epsilon, delta)-DP
    """
    # Aggregate client updates
    aggregated = sum(client_updates) / num_clients

    # Calculate noise scale for Gaussian mechanism
    sensitivity = 2.0 / num_clients  # Assuming clipped gradients
    sigma = sensitivity * math.sqrt(2 * math.log(1.25 / delta)) / epsilon

    # Add Gaussian noise
    noise = np.random.normal(0, sigma, aggregated.shape)
    private_aggregated = aggregated + noise

    return private_aggregated

Gradient Clipping

Gradient clipping is essential for differential privacy as it bounds the sensitivity of updates:

def clip_gradients(gradients, max_norm=1.0):
    """
    Clip gradients to bounded L2 norm

    This bounds sensitivity, enabling meaningful differential privacy
    """
    total_norm = 0.0
    for grad in gradients:
        param_norm = np.linalg.norm(grad)
        total_norm += param_norm ** 2
    total_norm = math.sqrt(total_norm)

    clip_coef = max_norm / (total_norm + 1e-6)

    if clip_coef < 1:
        for grad in gradients:
            grad *= clip_coef

    return gradients

Secure Multi-Party Computation (SMPC)

Core Principles

Secure Multi-Party Computation allows multiple parties to jointly compute a function over their inputs while keeping those inputs private. In federated learning, SMPC enables secure aggregation where the server learns only the aggregate result, not individual contributions.

Secret Sharing

Secret sharing splits a value into shares distributed across multiple parties. The original value can only be reconstructed when a threshold of shares is combined:

def shamir_share(secret, num_shares, threshold):
    """
    Shamir's Secret Sharing: split secret into shares

    Args:
        secret: Value to share
        num_shares: Total number of shares to create
        threshold: Minimum shares needed to reconstruct

    Returns:
        List of shares
    """
    import random

    # Generate random polynomial coefficients
    coefficients = [secret] + [random.randint(0, 2**32)
                               for _ in range(threshold - 1)]

    def evaluate_polynomial(x):
        result = 0
        for i, coef in enumerate(coefficients):
            result += coef * (x ** i)
        return result

    # Generate shares
    shares = [(i, evaluate_polynomial(i))
              for i in range(1, num_shares + 1)]

    return shares


def shamir_reconstruct(shares):
    """
    Reconstruct secret from shares using Lagrange interpolation
    """
    def lagrange_basis(j, x_values):
        def basis(x):
            numerator = denominator = 1
            for m, x_m in enumerate(x_values):
                if m != j:
                    numerator *= (x - x_m)
                    denominator *= (x_values[j] - x_m)
            return numerator / denominator
        return basis

    x_values = [x for x, _ in shares]
    y_values = [y for _, y in shares]

    # Evaluate at x=0 to get secret
    secret = 0
    for j in range(len(shares)):
        secret += y_values[j] * lagrange_basis(j, x_values)(0)

    return int(round(secret))

Secure Aggregation Protocol

A practical secure aggregation protocol for federated learning:

1. Setup: Server and clients agree on cryptographic parameters
2. Share Keys: Each pair of clients establishes a shared secret key
3. Mask Generation: Each client generates random masks using shared keys
4. Masked Upload: Clients add masks to their updates and upload
5. Aggregation: Server sums all masked updates (masks cancel out)
6. Result: Server obtains sum of client updates without seeing individuals

class SecureAggregationClient:
    def __init__(self, client_id, num_clients):
        self.id = client_id
        self.num_clients = num_clients
        self.shared_keys = {}

    def setup_pairwise_keys(self, other_clients):
        """Establish shared keys with other clients using Diffie-Hellman"""
        from cryptography.hazmat.primitives.asymmetric import dh
        from cryptography.hazmat.primitives.kdf.hkdf import HKDF
        from cryptography.hazmat.primitives import hashes

        for other_id in other_clients:
            if other_id == self.id:
                continue

            # Simplified: In practice, use proper DH key exchange
            shared_secret = hash((min(self.id, other_id),
                                 max(self.id, other_id)))
            self.shared_keys[other_id] = shared_secret

    def generate_mask(self, dimension):
        """Generate mask from pairwise keys"""
        mask = np.zeros(dimension)

        for other_id, key in self.shared_keys.items():
            # Generate pseudorandom mask from shared key
            np.random.seed(key)
            pairwise_mask = np.random.randn(dimension)

            # Add or subtract based on ID ordering
            if self.id < other_id:
                mask += pairwise_mask
            else:
                mask -= pairwise_mask

        return mask

    def create_masked_update(self, gradient):
        """Add mask to gradient for secure aggregation"""
        mask = self.generate_mask(len(gradient))
        return gradient + mask

Homomorphic Encryption

Introduction

Homomorphic encryption allows computations on encrypted data without decrypting it. For federated learning, this enables the server to aggregate encrypted model updates.

Types of Homomorphic Encryption

• Partially Homomorphic: Supports one operation (e.g., addition or multiplication)
• Somewhat Homomorphic: Limited number of operations
• Fully Homomorphic: Arbitrary computations (very expensive)

For federated learning aggregation, partially homomorphic encryption (PHE) supporting addition is sufficient:

from phe import paillier

class HomomorphicAggregator:
    def __init__(self):
        # Generate public/private key pair
        self.public_key, self.private_key = paillier.generate_paillier_keypair()

    def encrypt_update(self, update):
        """Client encrypts their update with public key"""
        encrypted = [self.public_key.encrypt(float(x)) for x in update]
        return encrypted

    def aggregate_encrypted(self, encrypted_updates):
        """
        Server aggregates encrypted updates without decryption

        Homomorphic property: Enc(a) + Enc(b) = Enc(a + b)
        """
        num_updates = len(encrypted_updates)
        dimension = len(encrypted_updates[0])

        # Initialize with first encrypted update
        aggregated = encrypted_updates[0]

        # Add remaining encrypted updates
        for i in range(1, num_updates):
            for j in range(dimension):
                aggregated[j] += encrypted_updates[i][j]

        # Divide by number of clients (scalar multiplication)
        aggregated = [x / num_updates for x in aggregated]

        return aggregated

    def decrypt_result(self, encrypted_aggregated):
        """Server decrypts final aggregated result"""
        decrypted = [self.private_key.decrypt(x)
                    for x in encrypted_aggregated]
        return np.array(decrypted)


# Usage example
aggregator = HomomorphicAggregator()

# Client 1
update1 = np.array([1.5, 2.3, -0.7])
enc1 = aggregator.encrypt_update(update1)

# Client 2
update2 = np.array([0.8, -1.2, 1.1])
enc2 = aggregator.encrypt_update(update2)

# Server aggregates without seeing individual updates
encrypted_avg = aggregator.aggregate_encrypted([enc1, enc2])
result = aggregator.decrypt_result(encrypted_avg)

print(f"Average: {result}")  # [1.15, 0.55, 0.2]

Trusted Execution Environments (TEE)

Hardware-Based Privacy

Trusted Execution Environments like Intel SGX and ARM TrustZone provide hardware-isolated secure enclaves where code and data are protected from the operating system and other applications.

TEE in Federated Learning

TEEs can be used for:

• Secure Aggregation: Server-side aggregation in isolated enclave
• Key Management: Secure storage of cryptographic keys
• Attestation: Clients verify server is running correct code
• Hybrid Approach: Combine TEE with cryptographic methods

Privacy Attacks and Defenses

Model Inversion Attacks

Attackers attempt to reconstruct training samples from model parameters or predictions. Defense strategies include:

• Differential privacy on model outputs
• Limiting prediction confidence scores
• Regularization techniques
• Ensemble methods that reduce overfitting

Membership Inference Attacks

Determining whether a specific data point was in the training set. Defenses:

• Differential privacy during training
• Regularization (dropout, L2)
• Prediction perturbation
• Confidence masking

Gradient Leakage Attacks

Recent research shows gradients can leak significant information. Example defense:

def defend_gradient_leakage(gradient, epsilon=1.0):
    """
    Combine multiple defense techniques against gradient attacks
    """
    # 1. Gradient clipping
    max_norm = 1.0
    grad_norm = np.linalg.norm(gradient)
    if grad_norm > max_norm:
        gradient = gradient * (max_norm / grad_norm)

    # 2. Add differential privacy noise
    sensitivity = max_norm
    scale = sensitivity / epsilon
    noise = np.random.laplace(0, scale, gradient.shape)
    gradient = gradient + noise

    # 3. Gradient compression (sparsification)
    k = int(0.1 * len(gradient))  # Keep top 10%
    threshold = np.sort(np.abs(gradient))[-k]
    gradient[np.abs(gradient) < threshold] = 0

    return gradient

Privacy-Utility Trade-offs

The Fundamental Trade-off

Stronger privacy guarantees typically reduce model utility (accuracy). Key factors affecting this trade-off:

Factor	Impact on Privacy	Impact on Utility
Lower ε (epsilon)	Stronger privacy	Lower accuracy (more noise)
Tighter clipping	Better privacy	Slower convergence
More participants	Better privacy (larger crowd)	Better utility (more data)
Fewer training rounds	Better privacy (less exposure)	Lower utility (undertrained)

Optimizing the Trade-off

def adaptive_privacy_budget(round_num, total_rounds, initial_epsilon=1.0):
    """
    Adaptively allocate privacy budget across training rounds

    Spend more budget in later rounds when model is converging
    """
    # Exponential budget allocation
    remaining_rounds = total_rounds - round_num
    budget_fraction = 1 - (remaining_rounds / total_rounds) ** 2

    epsilon = initial_epsilon * budget_fraction
    return max(epsilon, 0.1)  # Minimum epsilon


def privacy_accounting(epsilons, deltas, composition='advanced'):
    """
    Track cumulative privacy loss across multiple mechanisms

    Args:
        epsilons: List of epsilon values from different operations
        deltas: List of delta values
        composition: 'basic', 'advanced', or 'renyi'

    Returns:
        Total privacy budget
    """
    if composition == 'basic':
        # Simple composition: privacy losses add up
        total_epsilon = sum(epsilons)
        total_delta = sum(deltas)

    elif composition == 'advanced':
        # Advanced composition: sqrt improvement
        import math
        k = len(epsilons)
        epsilon = max(epsilons)
        delta = max(deltas)

        total_epsilon = epsilon * math.sqrt(2 * k * math.log(1/delta))
        total_delta = k * delta

    return total_epsilon, total_delta

Practical Implementation Guidelines

Choosing Privacy Techniques

Selection criteria for privacy methods:

Technique	Best For	Limitations
Differential Privacy	Strong mathematical guarantees	Accuracy loss, parameter tuning
Secure Aggregation	Hiding individual contributions	Communication overhead, dropout handling
Homomorphic Encryption	Complete encryption of updates	Very high computational cost
TEE	Hardware-backed security	Limited availability, side-channels

Combining Multiple Techniques

A robust federated learning system often combines multiple privacy techniques:

class PrivacyPreservingFederatedLearning:
    def __init__(self, epsilon=1.0, delta=1e-5, use_secure_agg=True):
        self.epsilon = epsilon
        self.delta = delta
        self.use_secure_agg = use_secure_agg
        self.privacy_budget_used = 0.0

    def client_update(self, local_data, global_model):
        # 1. Train locally
        gradients = self.train_local(local_data, global_model)

        # 2. Clip gradients (bound sensitivity)
        clipped = self.clip_gradients(gradients, max_norm=1.0)

        # 3. Add local differential privacy noise
        private_gradients = self.add_local_noise(clipped, self.epsilon)

        # 4. Apply secure aggregation masking (if enabled)
        if self.use_secure_agg:
            masked_gradients = self.apply_secure_mask(private_gradients)
            return masked_gradients

        return private_gradients

    def server_aggregate(self, client_updates):
        # 1. Aggregate (secure aggregation masks cancel out)
        aggregated = np.mean(client_updates, axis=0)

        # 2. Optional: Add central DP noise
        # (already have local DP, so this might be redundant)

        # 3. Track privacy budget
        self.privacy_budget_used += self.epsilon

        return aggregated