Chapter 5

Protocol Specification and Communication

Robust network protocols and communication patterns ensure reliable document exchange across agencies, systems, and borders, even in challenging network conditions with varying security requirements.

Transport Layer Security (TLS)

All public document API communications must use Transport Layer Security version 1.2 or higher. TLS provides confidentiality through encryption, integrity through message authentication codes, and authenticity through certificate validation. These properties are essential for protecting sensitive personal data in transit and ensuring communications originate from legitimate government systems.

TLS Configuration Requirements

Government document systems must enforce strong cryptographic parameters. TLS 1.3 is recommended as it eliminates vulnerable cipher suites and reduces handshake latency. Minimum acceptable is TLS 1.2 with approved cipher suites. Servers must support Perfect Forward Secrecy (PFS) using ECDHE or DHE key exchange, ensuring past communications remain secure even if long-term keys are compromised. Certificate validation must be strict—expired certificates, mismatched hostnames, and untrusted issuers must cause connection failures.

Mutual TLS (mTLS)

For system-to-system integrations between government agencies, mutual TLS provides two-way authentication. The server presents a certificate (as in normal TLS), but the client also presents a certificate. Both parties validate the other's certificate before establishing the connection. This ensures both ends of the communication are authorized government systems, preventing unauthorized access even if an attacker obtains API credentials.

1. Client initiates connection to server
   → Presents client certificate signed by government CA

2. Server validates client certificate:
   ✓ Signed by trusted CA
   ✓ Not expired
   ✓ Not revoked (OCSP/CRL check)
   ✓ Subject matches expected system identifier

3. Server presents server certificate

4. Client validates server certificate:
   ✓ Signed by trusted CA
   ✓ Hostname matches certificate CN/SAN
   ✓ Not expired
   ✓ Not revoked

5. If both validations succeed, TLS handshake completes
   → Encrypted channel established
   → Both parties authenticated

Message Format and Structure

Document exchange messages follow a standardized envelope format containing routing metadata, security credentials, and payload data. This separation allows intermediary systems to route and log messages without accessing sensitive content.

Message Envelope Structure

The message envelope includes a unique message ID for tracking and deduplication, source and destination identifiers, correlation IDs linking related messages (request-response pairs), timestamps for ordering and freshness validation, message type identifying the operation, and content metadata describing the payload format and encoding.

{
  "envelope": {
    "messageId": "msg-7f3d4e5a-8b2c-4567-890a-bcdef1234567",
    "correlationId": "req-123e4567-e89b-12d3-a456-426614174000",
    "sourceSystem": {
      "country": "EE",
      "agency": "Ministry of Interior",
      "system": "civil-registry-01"
    },
    "destinationSystem": {
      "country": "EE",
      "agency": "Police and Border Guard",
      "system": "border-control-api"
    },
    "messageType": "DocumentVerificationRequest",
    "timestamp": "2025-01-15T10:30:45.123Z",
    "priority": "normal",
    "ttl": 300,
    "security": {
      "signatureAlgorithm": "ECDSA-P384-SHA384",
      "signature": "MEYCIQCxGvX...",
      "certificateThumbprint": "sha256:7f3d4e5a..."
    }
  },
  "payload": {
    "documentId": "EE-PASS-2024-987654",
    "verificationLevel": "full",
    "purpose": "border-entry",
    "requestingOfficer": {
      "badgeNumber": "BE-45678",
      "name": "Officer Mets",
      "location": "Tallinn Airport Terminal 2"
    }
  }
}

Payload Encryption

While TLS encrypts data in transit, end-to-end encryption at the application layer provides defense in depth. Even if TLS is compromised or terminated at intermediate proxies, payload encryption ensures only the intended recipient can access document data. JSON Web Encryption (JWE) provides a standardized format for encrypted JSON payloads, supporting various encryption algorithms and key management approaches.

Synchronous vs Asynchronous Communication

Public document systems employ both synchronous request-response and asynchronous messaging patterns, each suited to different operational requirements.

Synchronous Request-Response

Synchronous communication uses HTTP request-response where the client blocks waiting for the server's response. This pattern is simple and works well for fast operations like document retrieval and verification. The client knows immediately whether the operation succeeded or failed. However, long-running operations (document generation, extensive validation) can timeout, and the client must maintain the connection throughout the operation.

Asynchronous Messaging

Asynchronous messaging decouples request submission from result retrieval. The client submits a request and receives an acknowledgment with a tracking ID. The operation proceeds in the background. The client later polls for status or receives a callback/webhook when complete. This pattern suits long-running operations, batch processing, and scenarios where immediate results aren't required.

Message Queue Integration

Message queuing systems provide reliable asynchronous communication with guaranteed delivery, automatic retries, and ordering guarantees. They're essential for inter-agency integration where systems operate independently and network connectivity may be intermittent.

Queue-Based Document Exchange

When Agency A needs to share documents with Agency B, messages are published to a queue. Agency B's systems consume messages at their own pace. If Agency B's systems are temporarily unavailable, messages accumulate in the queue without loss. Dead letter queues capture messages that fail processing after multiple attempts, allowing manual investigation. Message acknowledgment ensures each message is processed exactly once (or at-least-once with idempotency).

Apache Kafka for Event Streaming

Apache Kafka provides durable, ordered, replayable event streams. Unlike traditional message queues where messages are deleted after consumption, Kafka retains messages for a configurable retention period. This enables multiple consumers to process the same stream at different paces, replay historical events for system recovery or analytics, and build real-time data pipelines. Kafka's partitioning provides both ordering (within partition) and scalability (across partitions).

Topic: gov.ee.documents.events
Partitions: 12 (for parallel processing)
Retention: 30 days
Replication: 3 (for fault tolerance)

Event Types:
- document.created
- document.updated
- document.signed
- document.verified
- document.revoked
- document.expired

Event Schema:
{
  "eventId": "evt-123...",
  "eventType": "document.created",
  "timestamp": "2025-01-15T10:30:45Z",
  "documentId": "EE-BC-2025-001234",
  "documentType": "BirthCertificate",
  "agencyId": "EE-MOI",
  "metadata": {...},
  "payload": {...}
}

Consumers:
- Analytics pipeline (real-time reporting)
- Audit system (compliance logging)
- Replication service (disaster recovery)
- Integration gateway (cross-border exchange)

Distributed Document Storage

Modern document systems distribute storage across multiple nodes and geographic regions for availability, durability, and performance. Several architectural patterns address different requirements.

Primary-Replica Replication

A primary node handles all writes, replicating changes to replica nodes. Replicas serve read traffic, distributing load. If the primary fails, a replica is promoted to primary. This provides read scalability and fault tolerance while maintaining strong consistency for writes. PostgreSQL, MySQL, and MongoDB all support primary-replica topologies.

Multi-Master Replication

Multiple nodes accept writes concurrently, replicating changes to each other. This provides write scalability and geographic distribution—each region can have a local master, reducing latency. However, conflict resolution is required when concurrent updates occur to the same document. Strategies include last-write-wins (based on timestamp), application-specific conflict resolution, or manual intervention for critical data.

Distributed Consensus

Systems like etcd and Consul use Raft or Paxos consensus algorithms to achieve agreement across nodes without a central authority. These provide strong consistency guarantees—all nodes see the same data at the same time. However, consensus requires majority agreement, so writes can fail if more than half the nodes are unavailable. This trade-off (consistency over availability) is acceptable for critical data like document issuance.

Blockchain Integration

Blockchain provides immutable audit trails for document lifecycle events. Rather than storing full documents on-chain (expensive and impractical for large documents), systems typically store document hashes or proofs. This allows later verification that a document existed in a specific state at a specific time, without central authority.

Anchoring Document Hashes

When a document is issued, the system computes a cryptographic hash and records it on a blockchain. The blockchain entry includes the hash, document ID, timestamp, and issuing agency. Later, anyone can recompute the document hash and verify it matches the on-chain record. If the document has been altered, the hashes won't match. This provides non-repudiation—the issuing agency cannot later deny issuing the document or claim it had different contents.

1. Document issued in traditional database
   documentId: "EE-BC-2025-001234"
   content: {...}
   signature: "MEYCIQCxGvX..."

2. Compute document hash
   hash = SHA256(canonicalized_content + signature)
   hash = "7f3d4e5a8b2c4567890abcdef1234567..."

3. Create blockchain transaction
   {
     "operation": "DOCUMENT_ISSUED",
     "documentId": "EE-BC-2025-001234",
     "documentType": "BirthCertificate",
     "documentHash": "7f3d4e5a...",
     "issuingAgency": "EE-MOI",
     "timestamp": "2025-01-15T10:30:45Z"
   }

4. Submit to blockchain
   txHash = blockchain.submit(transaction)
   → Transaction confirmed in block #1234567

5. Store blockchain reference in database
   document.blockchainTx = txHash
   document.blockchainBlock = 1234567
   document.blockchainTimestamp = "2025-01-15T10:31:00Z"

6. Later verification
   a) Retrieve document from database
   b) Recompute hash from content
   c) Retrieve blockchain record
   d) Compare hashes
   → If match: document authentic and unaltered
   → If mismatch: document has been tampered with

Hyperledger Fabric for Permissioned Networks

Public blockchains like Ethereum are permissionless—anyone can participate. For government document systems, permissioned blockchains provide better control. Hyperledger Fabric allows defining which organizations can participate, which can see what data (privacy), and what endorsement policies are required for transactions (e.g., "must be signed by 2 of 3 government agencies"). This matches government operational requirements better than public blockchains.

Cross-Border Document Exchange

Exchanging documents across national borders requires additional protocols addressing sovereignty, data protection, legal frameworks, and technical interoperability.

Federated Identity and Trust

Cross-border systems use federated identity where each country maintains its own identity systems but trusts identities issued by partner countries. The eIDAS regulation in Europe establishes this mutual recognition framework. Technical implementation uses SAML or OpenID Connect for identity federation, with each country operating as an identity provider for its citizens. Cross-border services accept authentication assertions from recognized foreign identity providers.

Data Sovereignty and Localization

Many jurisdictions require personal data to be stored within national borders or limit international transfers. Document systems must accommodate these requirements through data replication with consent, query federation (documents stay in source country, queries sent there), or proxy caching (metadata stored locally, full documents retrieved on-demand).

Scenario: Finnish border control verifying Estonian e-ID

1. Finnish officer scans Estonian e-ID
   → Card presents certificate and biometric data

2. Finnish system validates certificate chain
   → Verifies certificate issued by Estonian CA
   → Estonian CA cert trusted per eIDAS framework

3. Finnish system queries Estonian verification service
   GET https://api.gov.ee/v2/verify-identity
   Headers:
     Authorization: Bearer [Finnish-Gov-Token]
     X-Requesting-Country: FI
     X-Requesting-Agency: FI-Border-Guard
     X-Purpose: border-control
   Body:
     personalCode: "38905144321"
     certificateSerial: "5f7d3e..."

4. Estonian system validates query authorization
   ✓ Requester is authorized Finnish government system
   ✓ Purpose is legitimate per bilateral agreement
   ✓ Logs request for audit

5. Estonian system returns verification result
   {
     "valid": true,
     "status": "active",
     "notRevoked": true,
     "attributes": {
       "fullName": "Jaan Tamm",
       "dateOfBirth": "1989-05-14",
       "citizenship": "EE"
     },
     "verificationId": "ver-abc123...",
     "timestamp": "2025-01-15T10:30:45Z"
   }

6. Finnish system grants border entry
   → Logs transaction
   → Person crosses border

Summary

Review Questions

1. Why is TLS 1.3 preferred over TLS 1.2? What security improvements does it provide?
2. Explain mutual TLS (mTLS). When would you use it versus standard TLS?
3. What is the difference between synchronous and asynchronous communication? Give examples of when each is appropriate.
4. How do message queues provide reliable delivery? What happens if a consumer fails while processing a message?
5. Compare primary-replica replication to multi-master replication. What are the consistency trade-offs?
6. Explain blockchain anchoring for documents. Why store hashes instead of full documents?
7. How does federated identity enable cross-border document verification? What is the role of eIDAS?
8. What is data sovereignty and how do document systems accommodate it while enabling international exchange?