Chapter 8: Conformance Testing, Certification and the Future

8.1 What Conformance Means Here

"Conformant" is used loosely in aviation software: a system that exports a file resembling the published example, or a demo screen with a green tick, gets called conformant. Neither is a conformance claim in an auditor's sense. A conformance claim is a statement that a named implementation, at a named version, satisfies a named set of requirements from a named version of a specification, backed by evidence that a third party can check without taking the vendor's word for it. Everything in this chapter is about making each of those four names precise.

Normative text and non-normative annexes

WIA-SPACE-016 follows the common IETF convention in which requirement keywords such as MUST, MUST NOT, SHOULD and MAY carry defined meanings (BCP 14, that is RFC 2119 as clarified by RFC 8174). Only such sentences in normative sections create obligations. Each PHASE document opens with a short normative introduction that states its purpose, and ends with a group of annexes (E to I) that are explicitly labelled non-normative: they describe implementation experience, the adoption roadmap, how evidence is shaped, the versioning policy and interoperability profiles.

A careful reader will notice a tension. Annex G is declared non-normative, yet it says that implementations claiming conformance MUST run all test vectors in CI and that vendors MUST report negative results. The sensible reading is that the annex adds no technical requirements to the data format or the API but defines the rules of evidence for anyone who makes a claim. This book treats it that way and lists the wording as an open editorial issue in §8.7.

Three stability tiers

Annex F sorts the content of each PHASE into three tiers so that early implementers know how much weight each part can bear.

The third row matters most for readers of this book. Earlier chapters pointed out a fatigue-life helper that returns physically impossible values, an SDK whose request function returns mock data and a CLI that always reports success. Because the reference layer is explicitly not normative and is versioned on its own, it can be fixed without any change to what a conformant record or API must look like.

WIA conformance is not airworthiness approval

The word "certification" appears in two unrelated senses in this domain, and mixing them up is a real risk. In the airworthiness sense, only a competent authority such as the FAA, EASA or, in Korea, the Ministry of Land, Infrastructure and Transport (MOLIT) can approve a design, a production organisation or a maintenance release, and those approvals are recorded on documents such as a type certificate, an FAA Form 8130-3 or an EASA Form 1. In the standards sense, an implementation can be shown to conform to WIA-SPACE-016, possibly with the help of a certification body operating under ISO/IEC 17065 (the international standard for bodies that certify products, processes and services, which Annex E cites in connection with audit retention).

The two never substitute for each other: a conformant records system can faithfully store a record of an unairworthy part, and a WIA conformance statement cannot make any part airworthy. Two details in the reference material blur this line and should be read with care. The comment on Certification.standard in the SDK types gives "WIA-SPACE-016" as an example value, which is fine for recording a conformance statement but must never be used where an airworthiness basis such as CS-25 or a TSO is meant. And the simulator's test tab displays a "Certificate ID" after pressing a button; that identifier is decorative text in a teaching tool, not a certificate of anything.

What can honestly be claimed today

At version 1.0 the published material has three gaps that limit any claim. The normative sections do not yet assign stable identifiers to individual requirements, so there is nothing for a profile manifest or a test matrix to point to except section numbers. PHASE 2 refers to a machine-readable api/openapi.yaml that is not present in the published repository. And the test-vector directories that Annex G describes are not yet published either. An implementer can still assign its own requirement identifiers, keyed to section numbers and sentence text, and publish them with its evidence. But for now a claim should read like "conforms to the normative requirements of PHASE 1 v1.0 as enumerated in the attached requirement list", not simply "WIA-SPACE-016 certified".

8.2 Test Vectors

A test vector is a small, self-contained input together with the verdict a correct implementation must reach on it. Vectors are the most portable form of conformance evidence because they depend on no particular language, database or deployment. Two implementations that disagree on one vector have found a concrete interoperability defect that can be discussed without opinions.

Where vectors live

Annex G fixes the directory layout: every PHASE has its own folder under tests/phase-vectors/, and every normative requirement in that PHASE must have at least one positive vector (an input that must be accepted) and at least one negative vector (an input that must be rejected).

The specification does not define the file format of a vector. Until it does, a format that states its own expectation is the easiest to audit. The example below is a proposal used throughout this chapter, not text from the standard; the requirement identifier is likewise invented, for the reason given in §8.1.

{
  "vector": "p1-neg-inspection-conditional-without-next-date",
  "requirement": "P1-INSP-004",
  "target": "Inspection",
  "expect": "invalid",
  "why": "A conditional result must say when the item is looked at again",
  "instance": {
    "id": "insp-000912",
    "inspectionDate": "2026-03-18",
    "inspector": "Licensed engineer, stamp 4471",
    "method": "eddy-current",
    "result": "conditional",
    "findings": "Linear indication 1.2 mm at fastener hole 14, within repair manual limits"
  }
}

Note that the rule this vector tests, that a conditional result requires a nextInspectionDate, is not in the reference types, where the field is simply optional. It is the kind of requirement an implementer would reasonably add and should then declare.

Designing negative vectors that matter

Positive vectors are easy to write; negative vectors take more thought, and they are where aviation data systems actually fail. A good negative vector encodes a mistake someone could plausibly make in a hangar or a data migration.

The last three rows show why a schema alone is never a test suite: rules that compare fields, look at history or compare records need code, and their vectors must carry more than one document.

Running vectors in CI and reporting every result

Annex G requires that implementations claiming conformance run every vector in continuous integration and publish the resulting pass/fail matrix. Two further rules give that matrix its value. First, negative results must be reported with the same fidelity as positive ones: a vendor may not publish only the rows that passed. Second, a vector skipped without a recorded justification counts as a failure. Together these close the most common loophole in self-assessment: quietly no longer running inconvenient tests.

A failing row matched by a partial or excluded entry in the profile manifest (§8.4) is still an honest statement. What auditors cannot accept is a matrix and a manifest that disagree.

8.3 Evidence Package and Quarterly Recheck

Publishing evidence is what turns test results into a claim. Annex G defines the compliance package as a signed tarball with fixed contents. Chapter 7 used CycloneDX and in-toto to describe a physical component and its manufacturing evidence. The package in this section is different: it describes the software implementation that stores and exchanges such records.

Why Sigstore rather than a private certificate authority

Annex G chooses Sigstore so that consumers can verify provenance "without trusting a private CA". In practice this means three pieces working together. The signature is wrapped in a DSSE (Dead Simple Signing Envelope), which binds the signed bytes to a declared payload type so that a signature over one kind of document cannot be replayed as a signature over another. The signing certificate is short-lived and tied to an identity, typically the CI workload of the vendor's repository. And the signing event is written to Rekor, a public append-only transparency log, so a signature cannot be produced secretly. Verification needs only the package and the expected signer identity; no vendor-run key server has to be trusted or kept alive for decades.

Putting it into a pipeline

The following pipeline shows the whole path from vectors to a signed package. It is written as a plain shell script so it can be dropped into any CI system. The tool invocations are real (CycloneDX's npm generator, the Sigstore cosign CLI), but file names, the requirement list and the version numbers are examples.

#!/usr/bin/env bash
set -euo pipefail
VERSION="3.4.1"                      # implementation version (example)
OUT="evidence-${VERSION}"
mkdir -p "$OUT"

# 1. Run every vector; the runner writes a matrix and exits non-zero on any
#    failure or unjustified skip (see the runner in section 8.6).
#    A failing matrix is still packaged: negative results are published, not hidden.
VEC_STATUS=0
node tools/run-vectors.cjs tests/phase-vectors > "$OUT/vector-matrix.json" || VEC_STATUS=$?

# 2. Software bill of materials in CycloneDX 1.5
npx @cyclonedx/cyclonedx-npm --spec-version 1.5 \
    --output-file "$OUT/sbom.cdx.json"

# 3. Interface description, profile manifest, changelog mapping
cp api/openapi.yaml              "$OUT/"
cp conformance/profile.json      "$OUT/"
cp conformance/version-map.json  "$OUT/"

# 4. Manifest of digests, then a keyless Sigstore signature logged in Rekor
( cd "$OUT" && sha256sum -- * > MANIFEST.sha256 )
cosign sign-blob --yes --bundle "$OUT/MANIFEST.sigstore.json" \
    "$OUT/MANIFEST.sha256"

tar -czf "${OUT}.tar.gz" "$OUT"
exit "$VEC_STATUS"                   # the CI job still reports the failure

Verification on the consumer side reverses the process: unpack, run cosign verify-blob against the bundle while pinning the expected certificate identity and OIDC issuer, then recompute every digest in MANIFEST.sha256. Pinning the identity is not optional. A valid signature from an unexpected identity proves only that someone signed something.

Quarterly recheck

Annex G asks implementations to re-publish the evidence package every quarter even when their own source code has not changed. The reason is environmental drift. A new compiler, validator library or base image can change behaviour without any vendor code changing; JSON Schema libraries, for example, differ in whether format keywords are asserted, and an upgrade can flip that. Diffing two quarterly packages shows precisely what moved.

Retention

Annex E suggests that a 7-year retention window satisfies ISO/IEC 17065 audit expectations in most jurisdictions, and immediately adds that where a regulator requires longer retention the deployment policy must extend the window. In aviation that exception is the normal case for the records themselves. Operating rules in both the United States and Europe require certain records, such as the current status of life-limited parts and of airworthiness directive compliance, to be kept for as long as the aircraft or part remains in service and to travel with it when it changes hands. Treat seven years as the floor for conformance evidence, and keep component records, with the evidence of how they were validated, for the life of the part.

Checklist before publishing a claim

8.4 Interoperability Profiles

Deployments differ in scope: a repair station may only record inspections, while an operator needs full traceability history. Annex I's interoperability profile lets each be honest about its scope without pretending to full conformance.

The profile manifest

Every implementation publishes a JSON profile manifest that lists requirement identifiers from the PHASE with one of three statuses: supported, partial with a reason, or excluded with a justification. The manifest is signed with the same Sigstore identity as the SBOM, so a consumer can be sure the scope statement and the evidence came from the same place. PHASE 2 adds that implementations declare in their profile which of the five operation groups they support. The example below combines both.

{
  "profiles": [
    "WIA-P1-DATA-FORMAT-Edge-only/2",
    "WIA-P2-API-INTERFACE-Edge-only/1"
  ],
  "implementation": {
    "name": "example-repair-station-records",
    "version": "3.4.1",
    "containerDigest": "sha256:0f4c…(truncated)"
  },
  "phaseVersions": {
    "phase-1-data-format": "1.0.0",
    "phase-2-api-interface": "1.0.0"
  },
  "operationGroups": {
    "componentCatalog": "supported",
    "maintenanceEvent": "supported",
    "traceability": "partial",
    "certificationPackage": "excluded",
    "serviceBulletin": "supported"
  },
  "requirements": [
    { "id": "P1-ID-001", "status": "supported" },
    { "id": "P1-MAT-003", "status": "partial",
      "reason": "Density accepted in kg/m3 only; legacy g/cm3 values are rejected, not converted" },
    { "id": "P2-CERT-001", "status": "excluded",
      "justification": "Repair station does not submit certification packages to an authority" }
  ],
  "operationGroupNotes": {
    "traceability": "History available from 2019 onward; earlier records held on paper"
  }
}

The partial traceability entry is exactly what an operator needs to know before relying on a system for back-to-birth history. The exclusion of certificationPackage is not a weakness: a repair station does not submit type-certification evidence.

Federation

When several implementations cooperate, for example an operator and three contracted MROs exchanging maintenance events, Annex I asks the federation to publish an aggregated manifest that summarises both the union and the intersection of the members' profiles. The intersection tells a directory service what every member can do, so a request that must succeed anywhere, such as a traceability query for a part that may have visited any shop, is routed on the least common denominator. If one member only partially supports traceability, this becomes visible before a query silently returns an incomplete history.

Registry, names and versions

Common deployment shapes are given names in a public registry kept by the WIA working group; Annex I mentions "Edge-only" and "Federated-with-replay" as examples. Registry entries are immutable. If the community wants a different shape, it is added under a new name rather than by editing the old entry, so that a manifest signed years ago still means what it meant. Profile names carry a major version in the same SemVer sense as the PHASE documents: WIA-P1-DATA-FORMAT-Edge-only/2 asserts conformance to the second major version of that named profile, not the second deployment of an unversioned one.

When a deployment widens its scope, the earlier manifest remains valid and signed for the deprecation window of §8.5, so auditors can later reconstruct what a system claimed on the day it accepted a record.

8.5 Versioning and Deprecation Policy

Component records outlive software: a landing-gear strut may stay in service, through overhauls, longer than most records systems stay in production. The versioning policy must therefore serve both implementers and records owners.

Semantic versioning applied to this standard

Annex H adopts Semantic Versioning 2.0.0. A major version indicates a backwards-incompatible change to a normative requirement, a minor version adds normative requirements without breaking existing implementations, and a patch version is editorial only. The table applies those rules to changes that WIA-SPACE-016 is likely to see.

The last row hides a discrepancy in the annexes. Annex E says that renaming or removing fields "even in error payloads, MUST trigger a minor version bump", while Annex H defines backwards-incompatible changes as major. Under SemVer a rename is incompatible, so this book follows Annex H. The practical rule for implementers is simpler still: never rename in place; add the new field, deprecate the old one, and remove it only at a major version.

Whether adding an enum value is compatible depends on consumers, not the schema alone: if they reject unknown values, a new value breaks them; if they must preserve and flag it, it does not. A future rule such as "unknown enumeration values MUST be preserved and MUST NOT cause the whole record to be rejected" would let the NDI-method and material enums grow through minor releases.

Parallel window and sunset

When a new major version is published, Annex H requires the previous major version to be maintained in parallel for at least 180 days, during which both are marked Stable in the WIA registry and either may be cited as WIA-conformant. The old version then enters a 12-month sunset window during which the registry marks it Deprecated, with a migration note that points to the replacement requirements and explains why the change was made. Because a version cannot be Stable and Deprecated at once, the two windows must run one after the other, although the annex does not say so in so many words.

          v2.0.0 published
                |
 v1.x  Stable   |  Stable (parallel window, at least 180 days)  |  Deprecated (sunset, 12 months)  |  retired
 v2.x           |  Stable  --------------------------------------------------------------------------->
                |
             day 0                                           day 180+                          day 180+ + 12 months

Annex F, in the same documents, speaks of "at minimum 90 days of overlap with the prior major version on all WIA-published reference implementations". The two figures may have been meant for different things, the specification in Annex H and the reference implementations in Annex F, but as written they conflict. This book treats Annex H's 180 days as governing, since it is the annex dedicated to versioning, and recommends that implementers plan for the longer figure in both cases.

Errata and the changelog mapping

Patch-level errata need no deprecation window; they are kept in a public register, each entry signed by the working-group chair. Annex H also recommends that implementations publish a mapping from each PHASE version they support to the specific build, container digest or SDK version that satisfies it. With the quarterly package, it lets an auditor say, for any date, which code accepted records under which version of the rules.

For the records themselves, store the schema version with every record, for example as a $schema URI or metadata.wiaSchemaVersion, so that nobody reading it twenty years later has to guess which rules applied.

8.6 Practising with the CLI and the Simulator

The standard ships two hands-on tools, the shell script cli/aircraft-component.sh and the browser simulator at wiastandards.com. Both help with orientation; neither is a conformance tool in its current form.

What the CLI actually does

As a skeleton that reserves a command-line shape, this is harmless. In a CI pipeline it would be harmful: an empty file would "pass".

What the simulator shows

A result screen with no inputs, matrix, version or signature is the opposite of Annex G evidence, which makes it a good teaching prop: ask students what would have to appear on it before an auditor could rely on it. Their list will closely match Table 8-5.

Building a real validator

A real validator has two layers. The first is a JSON Schema 2020-12 document for each object, of the kind developed in Chapter 4, executed by a validator that asserts format keywords. The second is a small set of coded rules for everything a schema cannot express. The package manifest of the reference SDK already lists zod as a dependency, although the current source never imports it; either route works, but JSON Schema is the one the standard names, and it is language-neutral, which matters for vectors that must run in many implementations.

The runner below loads schemas, runs every vector file, applies cross-field rules and writes the matrix that §8.3 signs. It uses Ajv's 2020-12 build and the ajv-formats plug-in so that format: date is actually checked. The rule functions are examples of the kind discussed in Table 8-3.

const fs = require("fs");
const path = require("path");
const Ajv2020 = require("ajv/dist/2020");
const addFormats = require("ajv-formats");

const ajv = new Ajv2020({ allErrors: true, strict: true });
addFormats(ajv);                                   // assert format: date
const schemas = {
  AircraftComponent: ajv.compile(require("../schemas/aircraft-component.json")),
  Inspection:        ajv.compile(require("../schemas/inspection.json")),
  Certification:     ajv.compile(require("../schemas/certification.json"))
};

// Rules a schema cannot express (cross-field and history rules)
const rules = {
  Certification: (c) =>
    c.expiryDate && c.expiryDate < c.issueDate ? ["expiryDate before issueDate"] : [],
  Inspection: (i) => {
    const errs = [];
    if (i.result === "conditional" && !i.nextInspectionDate)
      errs.push("conditional result without nextInspectionDate");
    if (i.nextInspectionDate && i.nextInspectionDate <= i.inspectionDate)
      errs.push("nextInspectionDate not after inspectionDate");
    return errs;
  }
};

function walk(dir) {
  return fs.readdirSync(dir, { withFileTypes: true }).flatMap((e) =>
    e.isDirectory() ? walk(path.join(dir, e.name))
                    : e.name.endsWith(".json") ? [path.join(dir, e.name)] : []);
}

const rows = [];
for (const file of walk(process.argv[2])) {
  const v = JSON.parse(fs.readFileSync(file, "utf8"));
  if (v.skip) {
    rows.push({ vector: v.vector, requirement: v.requirement, expect: v.expect,
                verdict: v.skip.reason ? "skipped-justified" : "fail" });
    continue;
  }
  const validate = schemas[v.target];
  const schemaOk = validate(v.instance);
  const ruleErrs = schemaOk && rules[v.target] ? rules[v.target](v.instance) : [];
  const observed = schemaOk && ruleErrs.length === 0 ? "valid" : "invalid";
  rows.push({ vector: v.vector, requirement: v.requirement, expect: v.expect, observed,
              verdict: observed === v.expect ? "pass" : "fail",
              detail: schemaOk ? ruleErrs : validate.errors });
}

process.stdout.write(JSON.stringify(rows, null, 2) + "\n");
process.exit(rows.some((r) => r.verdict === "fail") ? 1 : 0);

Comparing dates as strings works here only because ISO 8601 calendar dates in the form YYYY-MM-DD sort lexically in time order and the schema has already rejected anything else. That dependency on the schema layer running first is intentional and is why the runner applies rules only to documents that passed it.

The CLI's validate command can then become an honest wrapper: wrap a single file as a positive vector, run it and return the runner's exit code.

8.7 The Future

Version 1.0 models a component as a mostly static record. Several developments push towards records that are richer, more often updated and more tightly bound to evidence; none requires abandoning the model, but each shows where it must grow.

Structural health monitoring and digital twins

Structural health monitoring places sensors, such as strain gauges, acoustic-emission sensors, ultrasonic guided-wave transducers or crack-detection sensors, permanently on a structure so that its condition can be assessed without disassembly. Where regulators accept it, monitoring can supplement or replace a scheduled inspection. For a records standard this raises two questions. First, an Inspection today has an inspector and a method from a five-value list; a monitored check has a sensor system, a software version and an algorithm, which need their own traceable identity. Second, sensor data is continuous and voluminous, and it should not be stored inside the component record. The likely pattern is a reference from the record to an external data set, with a digest so the record still proves what data it relied on.

A digital twin, a model of one serial-numbered part tracking its actual loading and damage, is the natural consumer of such data. The WIA record would be its identity and approval backbone: the twin computes, the record says which part and which approved data the computation belongs to.

Additively manufactured parts

For additively manufactured parts, properties depend heavily on the process: feedstock lot, machine, build layout, parameters, heat treatment and post-processing. FAA and EASA have both published guidance and research on qualifying such parts. For records, "material" becomes inseparable from "process": a future version will likely need a build identifier and a link to the build record alongside MaterialSpecification.

New materials

Two material families already in service are poorly represented by the version 1.0 enumeration. Aluminium-lithium alloys, such as 2050 and 2198, offer lower density and better stiffness than classic 2xxx and 7xxx alloys and are used in fuselage and wing structure. Thermoplastic composites, unlike the thermoset systems discussed in Chapter 2, can be re-melted and welded by induction, resistance or ultrasonic methods, which introduces new special processes, new inspection needs for weld lines, and new process records. Today an Al-Li part can only be recorded under an existing aluminium value with the real alloy in grade, which misstates its family, and a thermoplastic part only under the generic composite; precise enum values, added under §8.5, would make fleet-wide queries reliable.

Provenance through transparency logs

Proposals to put parts provenance "on a blockchain" appear regularly. Signatures in a public transparency log, which the standard already uses, deliver the property that matters, that nobody can quietly rewrite history, without a token, a consensus network or shared custody of data; only signatures and digests are public. Extending this from software evidence to release documents and maintenance events would make a suspected unapproved part much harder to disguise.

Open issues for version 1.x

The problems identified in this book form a concrete agenda. Items marked "editorial" can be fixed as errata; the others need releases under §8.5.

None of these undermines the idea behind the standard. The mechanisms of this chapter, test vectors, honest profiles, signed evidence and disciplined versioning, are exactly the tools for fixing them in public.

Chapter Summary

• A conformance claim names an implementation, its version, a requirement set and a specification version, with verifiable evidence. Requirements have no identifiers yet, so claims must enumerate what they cover.
• Annex F separates Stable, Provisional and Reference tiers. The SDK, simulator and vectors are Reference material and never override normative text.
• WIA conformance concerns data and software. It is not, and cannot stand in for, an airworthiness approval by an authority.
• Each requirement needs positive and negative vectors; every result is published, and an unjustified skip is a failure. Many rules need code as well as a schema.
• The evidence package bundles an SBOM, the OpenAPI document, the vector matrix and a Sigstore-signed manifest, and is re-published every quarter. Seven years is the floor for retention; component records follow the part for life.
• Profile manifests declare supported, partial and excluded requirements and operation groups; registry names are immutable and versioned.
• SemVer governs changes; a superseded major version gets at least 180 days of parallel support and then a 12-month sunset. The 90-day figure in Annex F and the "minor" classification of renames in Annex E conflict with Annex H.
• The reference CLI and simulator are orientation tools; a real validator combines JSON Schema 2020-12 with coded rules.