📦 Contents

Chapter 5: Communication Protocols and Real-Time Data Flow (PHASE 3)

Chapter 4 described what a WIA-ENE-057 platform exposes: resources, scores, alerts and reports. This chapter is about how the bytes move. A desertification monitoring network is unusual among data systems because its producers live in hostile places. A soil probe in the Sahel runs on a solar panel and a battery. A field officer may upload a survey from a motorbike parked where one bar of cellular signal appears. A satellite pipeline pushes hundreds of megabytes of raster data on a sixteen-day rhythm. Downstream, a ministry dashboard wants a live view, a restoration partner wants a signed notification when a threshold is crossed, and a GIS analyst wants to pull the same data into a desktop tool through open geospatial services.

PHASE 3 of the standard (the protocol specification) defines the conventions that let all of these parties talk to one another: the transport and security layers, the real-time WebSocket stream, MQTT topics for sensors, the raster ingest request, OGC and GeoJSON interoperability, event and webhook delivery, bulk uploads, caching, and the retry and versioning rules that keep the whole system stable when links fail. The chapter follows the spec section by section, but it also points out where the spec examples are loose or internally inconsistent, because those are exactly the places where two conforming implementations can still fail to interoperate. The third tab of the companion simulator (“3. Protocol”) demonstrates the API and WebSocket message shapes described here.

5.1 Protocol Stack and Design Principles

Five layers, several channels

The spec draws a five-layer stack: an application layer (RESTful API), a transport layer (HTTPS and WSS), a data-format layer (JSON and GeoJSON), an authentication layer (OAuth 2.0 with JWT access tokens) and the network layer (TCP/IP). In practice the application layer is not a single API but four channels that share the lower layers: REST for request/response access, WebSocket for pushed updates, MQTT for constrained devices, and OGC web services for GIS clients. What they have in common is more important than what separates them: every channel carries the same PHASE-1 records keyed by locationId, every channel runs over TLS, and every channel authenticates with credentials issued by the same authorization server.

 Producers                      Platform (WIA-ENE-057)                 Consumers
 ---------                      ----------------------                 ---------
 soil / weather sensors --MQTT (TLS)--> broker --+
 field apps (ODK)       --HTTPS POST----------->  |
 satellite pipeline     --HTTPS multipart------>  +--> validation --> record store
 partner systems        --HTTPS bulk upload---->  |        |               |
                                                  |        v               v
                                                  |   event bus ----> webhooks  --> partner servers
                                                  |        |                     
                                                  |        +--------> WSS stream --> dashboards
                                                  +-----------------> REST / OGC --> GIS, reports

 Layers used on every arrow:
   Application : REST resources, WebSocket messages, MQTT topics, OGC operations
   Transport   : HTTPS / WSS / MQTT over TLS (TLS 1.3 required, see 5.8)
   Data format : JSON records, GeoJSON features, GeoTIFF / NetCDF rasters
   Identity    : OAuth 2.0 access tokens (JWT), device credentials
   Network     : TCP/IP

Figure 5-1 is worth reading from left to right once. Sensor readings enter through an MQTT broker, field forms and satellite products through HTTPS, and partner archives through the chunked bulk-upload flow. Everything passes through the same validation step that Chapter 3 described, so a record that would be rejected over REST (an NDVI of 1.5, a latitude of 95) is also rejected when it arrives over MQTT or inside a bulk file. Accepted records land in the record store and raise events; the event bus fans them out to webhooks and to the WebSocket stream, while REST and OGC endpoints serve the stored state on demand.

The five principles

PHASE 3 §1.2 lists five principles. They read like generic API advice, but each has a concrete consequence in this domain.

Idempotency deserves a caution. HTTP defines GET, PUT and DELETE as idempotent methods, but POST is not, and most ingest in this standard is POST. A client that times out after sending a vegetation record cannot tell whether the server stored it. The safe design is to make the record identifier the idempotency key: the server treats a second POST carrying VEG-20251225-SAHEL-001 as an upsert or a no-op, never as a new row. Chapter 7 returns to this in the ingest pipeline.

The spec describes two ways to select a version: the URL path (/v1/…, /v2/…) and a vendor media type in the Accept header (application/vnd.wia-ene-057.v1+json). Supporting both is allowed, but the URL path should remain the primary mechanism because it is visible in logs, proxies and caches; media-type versioning is easy to lose when an intermediary rewrites headers.

5.2 WebSocket Streaming

Connection lifecycle

Dashboards and operations consoles need updates without polling. The standard provides a WebSocket endpoint at /v1/stream over WSS; the hostnames used in the spec (such as api.wia-ene-057.org) are illustrative, and each implementation publishes its own. The WebSocket protocol itself is defined in RFC 6455: the client sends an HTTP GET with Upgrade: websocket, the server answers 101 Switching Protocols, and from then on both sides exchange frames. WIA-ENE-057 layers a small JSON message protocol on top.

 Client                                   Server (wss://.../v1/stream)
   |  HTTP GET /v1/stream  Upgrade: websocket   |
   |------------------------------------------->|
   |<-------------------------------------------|  101 Switching Protocols
   |  {"type":"auth","token":"..."}             |
   |------------------------------------------->|  verify token, scopes
   |  {"type":"subscribe", subscriptionId,      |
   |    locations, dataTypes, filters}          |
   |------------------------------------------->|  register subscription
   |                                            |
   |<-------------------------------------------|  {"type":"data", messageId, payload}
   |<-------------------------------------------|  {"type":"data", ...}
   |  {"type":"ping"}                           |
   |------------------------------------------->|
   |<-------------------------------------------|  {"type":"pong"}
   |                                            |
   x  network drops (e.g. cellular handover)    |
   |  reconnect after backoff delay             |
   |------------------------------------------->|  new socket
   |  auth  -> subscribe (same subscriptionId)  |
   |------------------------------------------->|
   |  GET /api/v1/monitoring/... ?since=<last   |
   |  timestamp seen>   (REST gap fill)         |
   |------------------------------------------->|

Authentication happens inside the socket: the first message is {"type":"auth"} with an access token. This design exists because browser WebSocket APIs cannot set an Authorization header on the handshake. It has a consequence the spec leaves implicit: the server must treat an open but unauthenticated socket as untrusted, deliver nothing to it, and close it if no valid auth message arrives within a short deadline. RFC 6455 provides close code 1008 (policy violation) for this purpose. Tokens should not be placed in the query string, where they end up in access logs.

Message types

{ "type": "auth", "version": "1.0.0", "token": "eyJhbGciOiJSUzI1NiIs..." }

{
  "type": "subscribe",
  "subscriptionId": "SUB-20251225-001",
  "locations": ["SAHEL-REGION-001"],
  "dataTypes": ["vegetation", "soil", "alerts"],
  "filters": { "minSeverity": "medium", "metrics": ["ndvi", "soilMoisture"] }
}

{
  "type": "data",
  "messageId": "msg-20251225-001",
  "timestamp": "2025-12-25T10:00:00Z",
  "subscriptionId": "SUB-20251225-001",
  "dataType": "vegetation",
  "locationId": "SAHEL-REGION-001",
  "payload": { "ndvi": 0.35, "evi": 0.28, "change": -0.02, "trend": "declining" }
}

Two spec documents describe this stream and they do not agree in detail. PHASE 3 subscribes with a locations array and delivers a generic data envelope with a payload; PHASE 2 §9 subscribes with locationIds and shows a typed vegetation-update message with a data object. An implementation should pick one form as canonical, declare it in its profile manifest, and be tolerant on input: accept both locations and locationIds in subscribe, and have clients switch on type while ignoring types they do not know. Tolerant readers are what allow the message set to grow without breaking deployed dashboards.

Reconnection without data loss

A WebSocket is a live pipe, not a queue. Anything the server sends while the client is disconnected is gone. A robust client therefore does three things on every reconnect: it waits a randomized, exponentially growing delay before trying again, it re-sends auth and subscribe (reusing its subscriptionId so server-side logs remain continuous), and it fills the gap by asking the REST API for records newer than the last timestamp it processed. Because the gap fill and the live stream may overlap, the client deduplicates by messageId or by record id.

// Reconnecting stream client (sketch). Values such as the URL are illustrative.
type Msg = { type: string; messageId?: string; timestamp?: string; [k: string]: unknown };

export function openStream(url: string, getToken: () => Promise<string>,
                           onData: (m: Msg) => void) {
  let attempt = 0;
  let lastSeen: string | undefined;            // newest timestamp processed
  const seen = new Set<string>();              // recent messageIds (bounded in practice)

  const connect = async () => {
    const ws = new WebSocket(url);
    let heartbeat: ReturnType<typeof setInterval> | undefined;

    ws.onopen = async () => {
      ws.send(JSON.stringify({ type: "auth", version: "1.0.0", token: await getToken() }));
      ws.send(JSON.stringify({
        type: "subscribe", subscriptionId: "SUB-20251225-001",
        locations: ["SAHEL-REGION-001"], dataTypes: ["vegetation", "soil", "alerts"],
      }));
      attempt = 0;
      heartbeat = setInterval(() =>
        ws.send(JSON.stringify({ type: "ping", timestamp: new Date().toISOString() })), 30_000);
      if (lastSeen) void backfillSince(lastSeen);  // REST gap fill, see text
    };

    ws.onmessage = (ev) => {
      const m: Msg = JSON.parse(String(ev.data));
      if (m.type === "pong") return;
      if (m.messageId && seen.has(m.messageId)) return;   // duplicate after reconnect
      if (m.messageId) seen.add(m.messageId);
      if (m.timestamp && (!lastSeen || m.timestamp > lastSeen)) lastSeen = m.timestamp;
      onData(m);
    };

    ws.onclose = () => {
      clearInterval(heartbeat);
      const delay = Math.min(60_000, 1_000 * 2 ** attempt++) * (0.5 + Math.random() / 2);
      setTimeout(connect, delay);
    };
  };
  void connect();
}

declare function backfillSince(isoTimestamp: string): Promise<void>;

The sketch keeps its state (last seen timestamp, recent message IDs) in memory. A dashboard can afford that; a service that forwards stream data into another system should persist both, otherwise a process restart reopens the gap.

5.3 MQTT Sensor Ingestion

Why MQTT for field devices

Soil-moisture probes, rain gauges and weather masts send small messages frequently and spend most of their time asleep. HTTPS with a full TLS handshake per reading is expensive on such devices. MQTT, a publish/subscribe protocol standardized by OASIS (version 3.1.1, also published as ISO/IEC 20922, and version 5.0), keeps one long-lived connection to a broker, uses compact framing, and lets the broker handle fan-out to subscribers. The spec lists both 3.1.1 and 5.0; version 5.0 adds features that matter in the field, such as message expiry and reason codes on acknowledgements.

Topic structure

Topics follow the pattern wia-ene-057/{locationId}/{sensorType}/{metric}. Putting the location first means a subscriber can take everything for one site with a single wildcard, and access control can be granted per location subtree.

The broker should enforce that a device can only publish below its own location. A sensor credential for SENSOR-SOIL-001 that is allowed to write to wia-ene-057/# can spoof readings for every site in the network.

Device message

Topic:  wia-ene-057/SAHEL-REGION-001/soil/moisture      (QoS 1)

{
  "deviceId": "SENSOR-SOIL-001",
  "locationId": "SAHEL-REGION-001",
  "timestamp": "2025-12-25T10:00:00Z",
  "sensorType": "soil-moisture",
  "readings": { "moisture": 12.5, "temperature": 28.5, "depth": "0-10cm" },
  "metadata": { "battery": 85, "signalStrength": -65, "firmware": "1.2.3" }
}

The payload carries deviceId, locationId, timestamp, readings and device metadata (battery percentage, signal strength in dBm, firmware version). The metadata is not decoration. A soil-moisture series that drifts downward while the battery falls below a certain level is more likely a failing sensor than a drying soil, and firmware versions are the only way to find all readings produced by a firmware release with a calibration bug. The ingest service should keep metadata alongside the readings rather than stripping it. Note that the locationId appears both in the topic and in the body; the ingest service should reject a message whose two values disagree.

Choosing QoS

MQTT offers three delivery levels, and the spec maps them to message classes: QoS 0 (at most once) for routine telemetry, QoS 1 (at least once) for important measurements, and QoS 2 (exactly once) for critical alerts. Two facts about MQTT shape how these should be used. First, QoS applies to each hop separately (device to broker, broker to subscriber), not end to end. Second, QoS 1 can deliver duplicates by design. The practical rule is therefore: use QoS 1 for measurements that feed records, make the consumer idempotent on deviceId plus timestamp, and reserve QoS 2 for the rare messages where its extra round trips are worth the battery cost. "Exactly once" at the MQTT layer never removes the need for deduplication in the application.

Buffering at the edge

Remote gateways lose connectivity for hours or days. A gateway should write every reading to local storage first and publish from that buffer, deleting entries only after the broker acknowledges them. When the link returns, the backlog is published in timestamp order at a throttled rate, or, for long outages, packaged as a bulk upload (5.7). Readings keep their original measurement time; the platform must never stamp them with the arrival time. This is also why clock synchronization appears in the security section below.

5.4 Satellite Raster Ingestion

Satellite products arrive as rasters, not records. The spec defines a single ingest request: an HTTPS POST to /api/v1/ingest/satellite with multipart/form-data, carrying a JSON part named metadata and a file part holding a GeoTIFF. NetCDF is named as an alternative container, which suits multi-dimensional climate or soil-moisture products.

POST /api/v1/ingest/satellite HTTP/1.1
Authorization: Bearer <access-token>
Content-Type: multipart/form-data; boundary=wia-boundary

--wia-boundary
Content-Disposition: form-data; name="metadata"
Content-Type: application/json

{
  "locationId": "SAHEL-REGION-001",
  "satellite": "MODIS",
  "instrument": "MODIS-Terra",
  "product": "MOD13Q1",
  "acquisitionDate": "2025-12-25T10:30:00Z",
  "resolution": "250m",
  "bands": ["NDVI", "EVI"],
  "cloudCover": 5.2,
  "crs": "EPSG:4326"
}
--wia-boundary
Content-Disposition: form-data; name="file"; filename="SAHEL-001-20251225.tif"
Content-Type: image/tiff

<binary GeoTIFF bytes>
--wia-boundary--

The metadata fields tie the raster to the rest of the data model: locationId binds it to a registered site, satellite, instrument and product identify the source, bands lists the layers present, cloudCover supports quality filtering, and crs states the coordinate reference system of the file. Three details are easy to get wrong:

• Composites are periods, not instants. MOD13Q1 is a 16-day composite; each pixel may come from a different day in the window. A single acquisitionDate timestamp cannot describe that. Record the compositing period (start and end dates) in an extension field, and use the period end or the product's nominal date consistently.
• The CRS is what the file contains, not what the product was. MODIS land products are distributed on a sinusoidal grid. If the file declares EPSG:4326, it has been reprojected, and the pipeline that did so should be recorded, because reprojection resamples pixels.
• Scaled integers. MOD13Q1 stores NDVI and EVI as 16-bit integers with a scale factor of 0.0001. A value of 3500 in the file means 0.35. The ingest service must apply the scale before the PHASE-1 range check, otherwise every pixel fails validation.

Large rasters should not be pushed through a single request on a weak link. Either stage the file in object storage and submit a reference, or use the chunked bulk flow of 5.7 with per-chunk checksums.

5.5 OGC and GeoJSON Interoperability

Three OGC services

GIS clients such as QGIS and ArcGIS speak the Open Geospatial Consortium service standards natively. The spec exposes three: WMS 1.3.0 for rendered map images (for example a desertification-risk layer), WFS 2.0.0 for vector features (the monitoring points and polygons with their attributes), and WCS 2.0.1 for coverages, that is, the raster values themselves (an ndvi-coverage). Every service also answers GetCapabilities, which clients call first to learn the layers, coordinate systems and formats available.

# WMS 1.3.0, CRS EPSG:4326: BBOX is minLat,minLon,maxLat,maxLon
GET /wms?service=WMS&version=1.3.0&request=GetMap&layers=desertification-risk
        &styles=&crs=EPSG:4326&bbox=14.0,-5.0,15.0,-3.0
        &width=800&height=600&format=image/png

# The same window with CRS:84 (longitude first): minLon,minLat,maxLon,maxLat
GET /wms?service=WMS&version=1.3.0&request=GetMap&layers=desertification-risk
        &styles=&crs=CRS:84&bbox=-5.0,14.0,-3.0,15.0
        &width=800&height=600&format=image/png

# WFS 2.0.0 GetFeature (cql_filter is a server vendor parameter, see text)
GET /wfs?service=WFS&version=2.0.0&request=GetFeature
        &typeNames=wia-ene-057:vegetation-monitoring
        &outputFormat=application/json
        &cql_filter=locationId='SAHEL-REGION-001'

# WCS 2.0.1 GetCoverage, trimming by named axes
GET /wcs?service=WCS&version=2.0.1&request=GetCoverage&coverageId=ndvi-coverage
        &subset=Lat(14.0,15.0)&subset=Long(-5.0,-3.0)&format=image/tiff

Three remarks on these requests. The spec writes the WFS parameter as typeName, the WFS 1.x spelling; WFS 2.0.0 names it typeNames, which is the form used above, and a tolerant server accepts both. The WFS example uses cql_filter, which is a vendor parameter supported by some servers (GeoServer among them) rather than part of WFS 2.0 itself; the standard mechanism is the OGC Filter Encoding FILTER parameter. An implementation that relies on cql_filter should say so in its profile. Finally, the WCS request trims by axis labels Lat and Long; these labels are defined by each coverage's description, so clients should read them from DescribeCoverage rather than hard-code them.

GeoJSON

For web clients the platform returns locations as GeoJSON (RFC 7946). A feature has a geometry and free-form properties, where the WIA fields live. RFC 7946 fixes the coordinate order to longitude, latitude and the CRS to WGS 84; it removed the old crs member, so the crs block shown in the spec example is unnecessary and some strict parsers ignore or reject it. Polygon rings must be closed (first position repeated as the last) and exterior rings should wind counter-clockwise.

{
  "type": "FeatureCollection",
  "features": [{
    "type": "Feature",
    "id": "SAHEL-REGION-001",
    "geometry": {
      "type": "Polygon",
      "coordinates": [[
        [-4.0, 14.0], [-3.0, 14.0], [-3.0, 15.0], [-4.0, 15.0], [-4.0, 14.0]
      ]]
    },
    "properties": {
      "locationId": "SAHEL-REGION-001",
      "areaUnit": "hectares",
      "ndvi": 0.35,
      "lastUpdate": "2025-12-25T10:00:00Z"
    }
  }]
}

The axis-order trap

The same place can be written three different ways within this one chapter, and mixing them is the most common interoperability bug in geospatial systems. The WMS 1.3.0 request in the spec uses crs=EPSG:4326 with bbox=14.0,-5.0,15.0,-3.0. That is correct: for EPSG:4326, WMS 1.3.0 follows the axis order of the CRS definition, which is latitude first. The same request in WMS 1.1.1, or with CRS:84, expects longitude first. A client that sends the latitude-first box to a longitude-first service silently gets a map of the wrong part of the world, or an empty one.

A useful sanity test runs in every CI pipeline: convert the running example point through each output path and assert that it lands in the Sahel band (latitude between roughly 10 and 20 degrees north) rather than in Central Africa south of the equator, which is where a swapped pair (longitude 14.5, latitude −4.0) would put it. Named fields in the WIA record format are the safest representation, which is one reason PHASE 1 uses them.

5.6 Events, Webhooks and Signature Verification

Event types and envelope

Changes inside the platform are published as events. PHASE 3 §6.1 defines five types: data.created, data.updated, data.deleted, alert.triggered and assessment.completed. Every event uses one envelope:

{
  "eventId": "evt-20251225-001",
  "eventType": "data.created",
  "timestamp": "2025-12-25T10:00:00Z",
  "source": "wia-ene-057-monitoring",
  "version": "1.0.0",
  "data": {
    "resourceType": "vegetation-monitoring",
    "resourceId": "VEG-20251225-SAHEL-001",
    "locationId": "SAHEL-REGION-001",
    "action": "created",
    "payload": { "ndvi": 0.35, "evi": 0.28, "timestamp": "2025-12-25T10:00:00Z" }
  },
  "metadata": { "organizationId": "org-12345", "correlationId": "corr-abc123" }
}

The eventId is the unit of deduplication: deliveries can repeat, events cannot. The correlationId ties together everything caused by one upstream action, so that a single field upload can be traced through validation, record creation, trend recomputation and the alert it triggered. The spec's example uses a short resource identifier; the example above uses the PHASE-1 identifier pattern (VEG-20251225-SAHEL-001) so that the event can be joined to the stored record directly. The payload is a summary, not the full record; consumers that need the whole record fetch it by resourceId.

Webhook registration and delivery

A partner registers a webhook with a target URL, a list of event types, optional filters (location IDs, severities) and a shared secret. The platform then POSTs each matching event with three headers:

Verifying the signature correctly

The spec's verification snippet has the right idea (HMAC-SHA256 and a constant-time comparison) but two practical flaws. Node's crypto.timingSafeEqual throws an exception when its two buffers differ in length, so a malformed or truncated header crashes the handler instead of returning 401. And the snippet does not say what payload is. The HMAC must be computed over the exact bytes received. Frameworks that parse JSON and hand you an object invite you to re-serialize it, and any difference in whitespace, key order or number formatting produces a different digest. The receiver below keeps the raw body, checks lengths first, deduplicates on eventId, and acknowledges quickly before doing real work.

// Express receiver for WIA-ENE-057 webhooks (Node.js 18+).
const crypto = require('crypto');
const express = require('express');

const app = express();
const SECRET = process.env.WIA_WEBHOOK_SECRET;     // shared secret set at registration
const processed = new Set();                       // use a persistent store in production

function verifySignature(rawBody, header, secret) {
  if (typeof header !== 'string' || !header.startsWith('sha256=')) return false;
  const expected = Buffer.from(
    'sha256=' + crypto.createHmac('sha256', secret).update(rawBody).digest('hex'), 'utf8');
  const received = Buffer.from(header, 'utf8');
  // timingSafeEqual throws on unequal lengths, so check first.
  if (received.length !== expected.length) return false;
  return crypto.timingSafeEqual(received, expected);
}

// express.raw keeps the exact bytes; re-serialising parsed JSON would change them.
app.post('/webhooks/wia-ene-057', express.raw({ type: 'application/json' }), (req, res) => {
  if (!verifySignature(req.body, req.get('X-WIA-Signature'), SECRET)) {
    return res.status(401).end();
  }
  const event = JSON.parse(req.body.toString('utf8'));
  if (processed.has(event.eventId)) return res.status(200).end();   // duplicate delivery
  processed.add(event.eventId);

  res.status(202).end();            // acknowledge fast, work asynchronously
  setImmediate(() => handleEvent(req.get('X-WIA-Event'), event));
});

function handleEvent(type, event) { /* enqueue for processing */ }

app.listen(8080);

A few operational rules complete the picture. The receiver should answer within a few seconds and queue heavy processing, because the sender will treat slow responses as failures and retry. A 2xx for a duplicate is correct: the event was already handled. Secrets should be rotated on a schedule, and during rotation the receiver should accept signatures made with either the old or the new secret. Finally, the signature covers the body only; it does not stop an attacker who captured a valid delivery from replaying it. Deduplication by eventId blunts replay, and an implementation can go further by adding a signed timestamp to the scheme as an extension and rejecting stale deliveries.

5.7 Bulk Upload, Caching and Resilience

Chunked bulk upload

Historical archives, a season of field surveys, or the backlog of a gateway that has been offline for a week are too large for single requests. PHASE 3 §7 defines a four-step flow: initiate, upload chunks, complete, and poll status.

POST /api/v1/bulk/upload/initiate
{ "dataType": "vegetation-monitoring", "recordCount": 10000,
  "estimatedSize": 52428800, "format": "json" }

-> { "uploadId": "upload-20251225-001", "chunkSize": 5242880,
     "expiresAt": "2025-12-26T10:00:00Z",
     "uploadUrl": "/api/v1/bulk/upload/upload-20251225-001/chunk" }

POST /api/v1/bulk/upload/upload-20251225-001/chunk     (repeat for 1..10)
X-Chunk-Number: 1
X-Chunk-Total: 10
<5 MiB of bytes>

POST /api/v1/bulk/upload/upload-20251225-001/complete
{ "uploadId": "upload-20251225-001", "chunksReceived": 10,
  "checksums": ["<sha256 of chunk 1>", "<sha256 of chunk 2>", "..."] }

GET /api/v1/bulk/upload/upload-20251225-001/status
-> { "status": "processing",
     "progress": { "totalRecords": 10000, "processed": 7500, "successful": 7450,
                   "failed": 50, "percentage": 75.0 },
     "errors": [ { "recordNumber": 245, "error": "Invalid NDVI value: 1.5" } ] }

The numbers in the spec example are consistent and illustrate the mechanics well: an estimated 52,428,800 bytes (50 MiB) with a server-chosen chunkSize of 5,242,880 bytes (5 MiB) gives ten chunks, numbered with X-Chunk-Number and X-Chunk-Total. The complete call lists a SHA-256 digest per chunk, which lets the server detect a corrupted or missing chunk and ask for that chunk alone rather than the whole file. That property matters on satellite links where a 50 MiB transfer may take many attempts.

The status response reports progress per record, not per byte: 7,500 of 10,000 records processed, 7,450 accepted and 50 rejected, with an error list such as record 245, "Invalid NDVI value: 1.5". This is the same PHASE-1 validation rule applied in bulk. A partial failure is normal; the client should download the error list, fix or discard those records, and resubmit them in a new, smaller upload instead of repeating the whole batch. Uploads also expire (expiresAt), so a client that loses its connection for longer than the window must start again.

HTTP caching

Much monitoring data changes slowly: a vegetation record is replaced every sixteen days, an assessment when new data arrives. HTTP caching (RFC 9111) saves bandwidth on exactly the links where it is scarcest. The spec uses the standard tools:

• Cache-Control — public, max-age=3600, s-maxage=7200 for shareable reference data; private, no-cache, must-revalidate for tenant data that may be cached but must be revalidated; no-store for responses containing personal data or credentials.
• ETag / If-None-Match — the client sends the tag it holds; if the resource is unchanged the server answers 304 Not Modified with no body.
• Last-Modified / If-Modified-Since — the date-based equivalent, coarser but simpler.

A field tablet that re-opens its site list each morning and receives 304s for unchanged locations spends almost nothing. Caching must, however, respect authorization: a response that depends on the caller's token is never public.

Retries, backoff and circuit breakers

The spec's retry rule is exponential backoff, delay = min(maxDelay, baseDelay · 2ⁿ), which with a one-second base gives 1, 2, 4, 8 and 16 seconds for five attempts. Two refinements are standard practice. Add random jitter, so that a hundred gateways that lost the same cellular tower do not all retry at the same instant when it comes back. And obey the server: a 429 Too Many Requests or 503 Service Unavailable with Retry-After (either a number of seconds or an HTTP date, per RFC 9110) overrides the client's own schedule.

A circuit breaker protects both sides when a dependency is down. In the closed state requests flow normally; when failures reach a threshold within the monitoring period the breaker opens and calls fail immediately without touching the network; after a reset timeout it goes half-open and allows a probe, closing again on success. The spec's configuration is failureThreshold 5, timeout 30,000 ms (per-request), resetTimeout 60,000 ms and monitoringPeriod 10,000 ms.

config: base = 1 s, maxDelay = 60 s, maxAttempts = 5
        failureThreshold = 5, timeout = 30 s, resetTimeout = 60 s, monitoringPeriod = 10 s

function call(request):
    if breaker.state == OPEN:
        if now - breaker.openedAt < resetTimeout: fail fast ("circuit open")
        breaker.state = HALF_OPEN                  # allow one probe
    for attempt in 0 .. maxAttempts-1:
        response = send(request, timeout)
        if response is success:
            breaker.recordSuccess()                # HALF_OPEN -> CLOSED
            return response
        if not retryable(response, request):       # 4xx other than 408/429, or
            return response                        # non-idempotent without a key
        breaker.recordFailure()                    # >= failureThreshold within
        if breaker.state == OPEN: fail fast        # monitoringPeriod -> OPEN
        if response has Retry-After:
            wait(parse(Retry-After))               # seconds or HTTP-date
        else:
            wait(random(0.5, 1.0) * min(maxDelay, base * 2^attempt))
    fail ("retries exhausted")

Only retry what is safe to retry. GET, PUT and DELETE may be repeated; a POST should be retried only when the server can recognize the repeat, which for this standard means the record carries its own id or the upload carries its uploadId. Client errors other than 408 and 429 are not retried at all: resending an invalid record five times produces five rejections.

Deprecation signals

When a version is retired, the server announces it in-band with a Sunset header (RFC 8594) giving the retirement date, a Deprecation header, and a Link with rel="alternate" pointing at the replacement. The spec example writes Deprecation: true, a form used by earlier drafts; the published RFC 9745 defines the value as a date, and new implementations should emit that form while tolerating the older one on input. Clients should log these headers and alert their operators, since a gateway in a remote station may keep running unattended long after its API version is switched off. Chapter 8 covers the governance side of versioning, including the overlap windows.

5.8 Security at the Protocol Layer

Transport encryption

The two spec documents set different floors. PHASE 3 §5.1 says TLS 1.2 or higher with strong ciphers and perfect forward secrecy (ECDHE or DHE key exchange, AES-256-GCM or ChaCha20-Poly1305). The API-INTERFACE document §7 says implementations MUST encrypt in transit with TLS 1.3 (RFC 8446). The stricter statement should be treated as the requirement: TLS 1.3 on every public endpoint, including WSS and MQTT over TLS. TLS 1.3 provides forward secrecy by construction and has removed the weak cipher options, so the cipher list in PHASE 3 becomes relevant only where an implementation must still accept TLS 1.2 from legacy devices. If that exception is needed, it belongs in the profile manifest as a partial conformance with a stated reason, not as a silent default.

OAuth 2.0 and JWT access tokens

Chapter 4 showed how machine clients obtain tokens with the client-credentials grant. For applications acting on behalf of a person, such as a restoration manager signing into a web dashboard, PHASE 3 §5.2 describes the authorization code flow: redirect to /oauth/authorize with the requested scopes and a state value, receive a code at the registered callback, exchange it at /oauth/token, and receive an access token (3,600 seconds lifetime in the example) with a refresh token. The spec's flow omits PKCE (RFC 7636); current OAuth security guidance, RFC 9700 (the OAuth 2.0 Security Best Current Practice), recommends PKCE for all clients using the code flow, and a WIA deployment should require it.

Access tokens are JWTs (RFC 7519) signed with RS256. A resource server validates them without calling the authorization server: it looks up the public key named by kid in the published key set, verifies the signature, and then checks the claims.

Header : { "alg": "RS256", "typ": "JWT", "kid": "wia-ene-057-key-1" }

Payload: {
  "iss": "https://auth.example.org",
  "sub": "user-12345",
  "aud": "https://api.example.org",
  "iat": 1735124400,
  "exp": 1735128000,
  "scope": "monitoring:read restoration:write",
  "organization": "org-12345",
  "role": "project-manager"
}

The algorithm must be pinned: a verifier that accepts whatever alg the token header names can be tricked into accepting unsigned or wrongly signed tokens. The same token validation applies to the WebSocket auth message; when a token expires during a long-lived stream, the server should close the socket and let the client reconnect with a fresh token.

Clocks, storage and audit

Three further requirements come from the API-INTERFACE document. Clocks must be synchronized with NTPv4 (RFC 5905) at stratum 2 or better; this makes token expiry checks meaningful and keeps the ordering of events consistent across servers. On remote gateways without a reliable network time source, GPS receivers provide an accurate reference and are commonly used for this purpose. Data at rest must be encrypted with AES-256-GCM or stronger. And the platform must keep tamper-evident audit logs, built with a Merkle-tree structure in the style of transparency logs, so that the history of who changed which record cannot be quietly rewritten. The same document requires each deployment to declare the privacy regime that applies to personal data, such as GDPR or Korea's PIPA; in this domain that mainly concerns field collectors, whose names, phone numbers and GPS tracks travel with survey submissions.

A last word on the compliance list at the end of PHASE 3. It marks HTTP/1.1 (RFC 7230–7235), HTTP/2 (RFC 7540), OAuth 2.0 (RFC 6749) and others with check marks. These are self-declared alignments, not certifications, and several of the cited documents have since been replaced: HTTP semantics and caching are now RFC 9110 and RFC 9111, HTTP/1.1 messaging RFC 9112, and HTTP/2 RFC 9113. An implementation should cite the current documents in its own conformance statement.

Summary

PHASE 3 turns the data model and API into a working network. The points to carry forward are these: all channels share TLS, OAuth-issued credentials and PHASE-1 validation; the WebSocket stream needs in-band authentication, tolerant parsing and REST gap-filling on reconnect; MQTT topics are organized by location, with QoS 1 plus application-level deduplication as the default; raster metadata must describe composites, projections and scale factors honestly; axis order must be tested, not assumed; webhook signatures are verified over raw bytes with a length check before the constant-time compare; and retries are bounded, jittered and limited to operations the server can recognize as repeats. Chapter 6 takes these protocols outward, to the Earth-observation archives, land-degradation reporting processes, GIS platforms and carbon registries that a WIA-ENE-057 platform connects to.