Chapter 8
A claim that a system “implements WIA-BIO-004” is only worth something if someone else can check it. This chapter turns the standard’s requirements into a checklist, builds golden and negative test vectors with exact expected values, runs them against the v1.0 SDK and CLI, and records what passes, what fails and what the next revision should fix.
The previous seven chapters described what WIA-BIO-004 asks for: a vocabulary of biomarker types and clinical roles (Chapter 1), a fixed set of diagnostic statistics (Chapter 2), JSON records for measurements, reference ranges and validation studies (Chapter 3), an SDK and CLI (Chapter 4), analytical and clinical validation protocols (Chapters 5 and 6), and a privacy-aware integration architecture (Chapter 7). None of that helps a laboratory director, a sponsor’s data manager or a regulator who has to decide whether a particular product actually behaves as the standard says. For that you need a conformance claim: a statement of which requirements a system meets, backed by evidence that a third party can re-run.
Conformance testing for a data standard of this kind has an unusual property. Most of what WIA-BIO-004 specifies is arithmetic, and arithmetic has right answers. Sensitivity at a given cutoff is not a matter of opinion; neither is the trapezoidal area under a set of ROC points. That makes it possible to write test vectors whose expected outputs are known to four decimal places, and to find real defects with them. We do exactly that in this chapter, and several of the results are uncomfortable: the v1.0 reference SDK does not type-check as shipped, under-reports AUC on tied or sparse threshold grids, and the v1.0 CLI prints fixed demonstration numbers regardless of its input. Recording such findings honestly is the point of conformance work, not an embarrassment to be hidden.
All SDK observations below were made on a scratch copy of the v1.0 TypeScript sources (api/typescript/src/index.ts and types.ts), compiled with TypeScript 5.9 and run under Node.js 22. Where we had to change the source to get it to compile, we say so.
WIA-BIO-004 v1.0 does not contain a single section titled “conformance”. The requirements that a conformance claim can be tested against are spread over four places, and a checklist has to gather them:
"standard": "WIA-BIO-004" envelope and a declared version; a conformant consumer accepts them.B001 (insufficient sample size) to B006 (low statistical power); the SDK extends the set to B010. A system that reports failures must use these codes with their stated meanings.The first three are properties of software and data; the fourth is a property of a biomarker study. They must be kept apart in a claim. A vendor’s statistics engine can be fully conformant while the biomarker a customer validates with it fails requirement 3 because only one cohort was ever recruited. Conversely, a well-designed study run on non-conformant software may still produce correct numbers; it simply cannot prove that it did.
Equally important is what the documents do not say. v1.0 defines no numbered conformance levels, no tiers and no pass-rate thresholds. Tiered schemes of the “bronze / silver / gold” kind, familiar from other certification programmes, have no basis in v1.0, and this book does not invent one. The README links a certification site, but the v1.0 documents do not describe a certification procedure, an accredited assessor or a test suite. The phase annexes describe a non-normative idea of an evidence package and a profile manifest (§8.4 below), and refer to vector directories that are not present in the repository.
The practical consequence is that a conformance claim under v1.0 is a self-declaration with evidence. The strongest form of it lists every requirement, marks each one as supported, partially supported or excluded, and attaches reproducible test results. The rest of this chapter shows how to build that.
The following table is the checklist used throughout the chapter. The identifiers (C-01 …) are this book’s, introduced so that test results and manifests can refer to a row; they are not numbered in the standard.
| ID | Requirement (source) | Evidence | How tested |
|---|---|---|---|
| C-01 | Standardised measurement capture (§8.1-1, §7.1) | Sample §7.1 records produced by the system | Schema validation; field-by-field comparison with a known input |
| C-02 | Automated QC checks (§8.1-2) | QC rule configuration; audit of flagged samples | Feed records with qualityFlags.hemolyzed = true; confirm exclusion or annotation |
| C-03 | ROC, sensitivity, specificity (§8.1-3, §4) | Output of the statistics engine | Golden vectors G-01 to G-07 (§8.2) |
| C-04 | Clinical correlation with outcomes (§8.1-4, §6) | Correlation / survival analysis outputs | Reference data set with a known Spearman ρ or hazard ratio computed by an independent package |
| C-05 | Standardised performance report (§8.1-5, §5.3.1) | Generated reports | Internal consistency: PPV/NPV must follow from Sens, Spec and the stated prevalence |
| C-06 | Record envelope and version (§7.1, §7.2) | Every emitted record | standard equals "WIA-BIO-004"; version is a semantic version |
| C-07 | Unit consistency (§7.1, Chapter 3) | Validator behaviour | Negative vector N-05 (value unit differs from reference-range unit) |
| C-08 | Temporal consistency (Chapter 3) | Validator behaviour | Negative vector N-06 (dateMeasured earlier than dateCollected) |
| C-09 | Error codes B001–B006 (§8.3) | Error responses | Negative vectors N-01 to N-04 must return the documented codes |
| C-10 | Sample-size requirement (README 1) | Study record: population.cases, controls | Check against the stated minimum; note the README / SDK difference (§8.3) |
| C-11 | Independent cohorts (README 3) | validation.validationCohorts[] | At least two independent populations, each validation cohort listed with its own AUC (see §6.8) |
| C-12 | Blinding and reference standard (README 4–5) | referenceStandard.method, protocol document | Document review; cannot be verified from data alone |
| C-13 | Reproducibility and stability (README 6–7) | Precision and stability reports (Chapter 5) | Recompute CV from raw replicate data; compare with the acceptance limit the claim names |
| C-14 | Privacy controls (§9) | De-identification and encryption configuration | Inspection; re-identification spot checks (Chapter 7) |
Rows C-12 to C-14 are not automatable in the sense that C-03 is. They still belong in the checklist, because a claim that silently leaves them out overstates what was checked.
A test vector is an input together with its expected output and a tolerance. Golden vectors exercise normal behaviour and must produce specific numbers; negative vectors are inputs that a conformant system must reject, with a specific error. Both kinds are needed. A statistics engine that returns plausible numbers for every input, including invalid ones, passes every golden vector and is still dangerous.
The cheapest golden vectors are the worked examples already printed in the specification, because anyone can check them by hand. Two need care. The §5.3.1 report example quotes PPV 86.0 % and NPV 84.5 %, which do not follow from its own sensitivity (82 %), specificity (88 %) and prevalence (0.50); the vector below uses the recomputed values. And Appendix A.1 gives only the counts at one threshold, not the underlying measurements, so to test ROC behaviour we need a full data set, which §8.2.2 supplies.
| Vector | Input | Expected output | Source |
|---|---|---|---|
| G-01 | TP 75, FN 10 | Sens = 88.2 % | §4.1.1 |
| G-02 | TN 95, FP 5 | Spec = 95.0 % | §4.1.2 |
| G-03 | TP 42, FN 8, TN 45, FP 5 | Sens 84.0 %, Spec 90.0 %, PPV 89.4 %, NPV 84.9 %, Acc 87.0 %, LR+ 8.40, LR− 0.178, J 0.74 | Appendix A.1 |
| G-04 | Sens 84 %, Spec 90 %, prevalence 0.10 | PPV 48.3 %, NPV 98.1 % | §4.1.3 (Bayes form) |
| G-05 | Sens 82 %, Spec 88 %, prevalence 0.50 | PPV 87.2 %, NPV 83.0 %, LR+ 6.83, LR− 0.20 | §5.3.1, corrected |
| G-06 | AUC 0.87, n₁ = n₀ = 150 | Hanley–McNeil SE 0.0210; 95 % CI 0.829–0.911 | §10.3 |
| G-07 | Synthetic set S-100 (§8.2.2), cutoff 75 | Confusion matrix 42 / 8 / 45 / 5; exact AUC 0.9358; optimum J = 0.74 | This book |
G-06 deserves a comment. The §7.2 validation-study example reports AUC 0.87 with a 95 % CI of 0.82–0.92 for 150 cases and 150 controls. The Hanley–McNeil formula printed in §10.3 gives 0.829–0.911 for the same inputs. The difference does not mean the example is wrong, since a bootstrap or DeLong interval can legitimately be wider, but it does mean a validator should not reject a CI merely because it differs from one formula. What a validator should require is that the record states how the interval was obtained, which v1.0 records have no field for. That gap reappears in §8.7.
To test ROC code we need individual values, not counts. The following set, called S-100 here, is constructed so that the cutoff “value ≥ 75 is positive” reproduces Appendix A.1’s counts exactly. It is entirely synthetic and describes no real assay; its virtue is that anyone can regenerate it in one line of any language.
S-100 (integers, inclusive ranges; label 1 = case, 0 = control)
cases (50): 76, 77, …, 117 → 42 values, all ≥ 75 (TP at cutoff 75)
60, 61, …, 67 → 8 values, all < 75 (FN)
controls (50): 30, 31, …, 74 → 45 values, all < 75 (TN)
80, 81, …, 84 → 5 values, all ≥ 75 (FP)
Cutoff 75 : TP 42 FN 8 TN 45 FP 5 (= Appendix A.1)
Ties : case values 80–84 equal control values 80–84 (5 tied pairs)
Exact AUC : P(case > control) + ½·P(tie) over 2,500 pairs = 0.9358
Youden optimum : J = 0.84 + 0.90 − 1 = 0.74, first reached at observed value 76
Hanley–McNeil : SE(0.9358; 50, 50) = 0.0258 → 95 % CI 0.885–0.986Two properties make S-100 useful beyond G-03. It contains ties between groups, which is where ROC implementations most often go wrong, and its cutoff of 75 is not one of the observed values, so it also tests how an implementation labels the optimal threshold. An engine that reports the optimum as 76 (the smallest observed value that yields the same classification) and one that reports 75 are both correct, provided the reported sensitivity and specificity are 0.84 and 0.90. A test should therefore assert on the operating point, not on the threshold label.
Negative vectors pin down the error behaviour promised in §8.3 and the record-level rules from Chapter 3. The “expected” column is what a conformant implementation should do; §8.3 reports what v1.0 actually does.
| Vector | Input | Expected behaviour |
|---|---|---|
| N-01 | S-100 with the last value of predictions removed (label/value length mismatch) | Reject with B002 (invalid measurement values) |
| N-02 | S-100 with every label set to 1 | Reject with B002: both classes are needed |
| N-03 | First 99 values of S-100 | Reject with B001 (insufficient sample size) |
| N-04 | Validation study with no referenceStandard | Reject with B003 (missing reference standard) |
| N-05 | §7.1 CRP record with unit: "mg/dL" but referenceRange.unit: "mg/L" | Reject or normalise explicitly via UCUM; never compare raw numbers |
| N-06 | §7.1 record with dateMeasured earlier than dateCollected | Reject as a data-quality error |
| N-07 | Measurements whose diseaseStatus is the string "healthy" instead of a boolean | Reject; strings must never be coerced to “diseased” |
| N-08 | Request for a 90 % confidence interval | Use z = 1.645, or reject the unsupported level; never silently use another z |
N-04 has no counterpart in the SDK, whose inputs contain no reference-standard field; it applies to systems that accept §7.2 records. N-05 and N-06 are checks that belong in a record validator in front of the statistics engine rather than in the engine itself.
Expected values need tolerances, and the tolerances should reflect how the numbers are printed, not how generous the tester feels. The standard prints percentages to one decimal and likelihood ratios to two, so a sensible rule is: counts must match exactly; percentages within ±0.05 percentage points of the unrounded value; ratios and AUC within ±0.0005. A tolerance is not an allowance for a different formula. If an implementation uses a different CI method, the correct response is to record the method, not to widen the tolerance until the vector passes.
We ran the vectors above against the v1.0 SDK. The results fall into three groups: things that pass, behaviours that are correct but easy to misuse (and must be documented in any claim), and outright defects.
The first test of any SDK is whether it builds. The v1.0 sources do not type-check with TypeScript 5.9: types.ts ends with an export type { … } block and an export { BIOMARKER_CONSTANTS, BiomarkerErrorCode, BiomarkerError } line that re-export names already exported at their declarations, and index.ts repeats export { BiomarkerDataSDK } for a class declared with export class. The compiler reports TS2323 (“cannot redeclare exported variable”) and TS2484 (“export declaration conflicts with exported declaration”). Removing the redundant blocks, which changes no behaviour, lets the package compile. Everything below was measured on that corrected copy. The package also declares a Jest configuration but ships no test files, so there is no in-repository evidence of the kind this chapter builds.
With a fixed cutoff of 75 and the default prevalence of 0.5, validateBiomarker on S-100 returns the confusion matrix 42 / 45 / 5 / 8, sensitivity 84, specificity 90, accuracy 87, PPV 89.36, NPV 84.91, LR+ 8.40 and LR− 0.178, which matches G-03. N-01 and N-02 throw BiomarkerError with code B002; N-03 throws B001. The Hanley–McNeil standard error is implemented exactly as printed in §10.3. The Youden optimum on the automatic grid is reported at value 76 with sensitivity 0.84, specificity 0.90 and J = 0.74, the operating point G-07 expects.
Two scales. DiagnosticPerformance reports sensitivity, specificity, PPV, NPV and accuracy in percent, while every ROCPoint and optimalThreshold reports sensitivity and specificity as fractions from 0 to 1. A test that asserts 84 against optimalThreshold.sensitivity fails; a dashboard that mixes the two shows 0.84 % sensitivity. Conformance tests should assert the scale explicitly.
Prevalence-based predictive values. The SDK never computes PPV as TP/(TP + FP). It always applies the Bayes form with the prevalence argument, default 0.5. On S-100 the two agree only because the sample is balanced. With prevalence: 0.1 the same data give PPV 48.3 % and NPV 98.1 % (G-04), while the count-based values remain 89.4 % and 84.9 %. Both are legitimate numbers answering different questions; a report must say which one it contains and at what prevalence.
The 100-sample floor. The README asks for “100 cases and 100 controls”; the SDK constant MIN_VALIDATION_SAMPLES is 100 in total. A study with 60 cases and 40 controls passes the SDK and fails the README. Also, because validateBiomarker always calls calculateROC internally, a 99-sample input throws B001 instead of returning a result with errors[] filled in; callers expecting the errors array never see it.
AUC under-estimation with ties in false-positive rate. calculateAUCTrapezoidal sorts ROC points by FPR only. When several points share an FPR, they stay in threshold order, which means descending TPR, and the trapezoid joins the wrong corners of each vertical run. On S-100 the SDK returns AUC 0.9294 where the exact value is 0.9358. Adding TPR as a secondary sort key (ascending) restores 0.9358. The error is not caused by the tied values alone: after removing the ties (shifting the five control values to 80.5–84.5), the SDK still reports 0.9284 against an exact 0.9348.
User thresholds without end points. When the caller supplies thresholds, the SDK evaluates only those, and does not add the (0, 0) and (1, 1) corners. With thresholds: [70, 75, 80] on S-100 the reported AUC is 0.08, not an approximation of 0.94 but a meaningless number, because the only area integrated is the thin strip between three points. The README’s examples pass short threshold lists exactly like this. A conformant engine must either always add the corners or refuse to report an AUC from a partial grid.
Confidence level. Any confidenceLevel other than 0.95 silently uses z = 2.576. A request for a 90 % interval on S-100 therefore returns 0.860–0.999, which is a 99 % interval and wider than the correct 90 % interval of 0.885–0.974 at the SDK’s own AUC. The source comment calls the method a “DeLong approximation”; it is the Hanley–McNeil formula, and a claim must not describe it as DeLong.
No unit check. validateBiomarker accepted S-100 with half of the measurements labelled ug/L and the other half ng/mL without a warning. The two happen to be the same quantity, but the SDK would have done the same for mg/L. Unit consistency (C-07) must therefore be enforced by a record validator before data reach the SDK.
String disease status. The SDK splits groups with strict comparisons (=== true, === false) but builds its labels with a truthiness test, so any non-empty string counts as diseased. If every status is a string, both groups come out empty and the call ends with B002, which is safe for the wrong reason. If the input mixes booleans and strings, the result is silently wrong: marking ten of the S-100 controls "healthy" instead of false turns them into cases, the confusion matrix becomes 42 / 35 / 5 / 18, sensitivity drops to 70 %, and the call still returns isValid: true with an empty errors array.
The v1.0 CLI parses its options (for calc-roc it also checks that the data file exists and counts its rows), then prints fixed demonstration output. Given a one-line CSV file, calc-roc reports “Total samples: 1” followed by a confusion matrix of 75 / 85 / 15 / 25 and an AUC of 0.85. The CLI has no --version flag, although the README’s quick-start uses it; the command is wia-bio-004 version. Until the CLI reads its input, it cannot be part of a conformance claim, and no number it prints should appear in evidence.
| Finding | Vector | Expected | v1.0 result | Status |
|---|---|---|---|---|
| F-01 Build | — | Compiles | TS2323 / TS2484 | Fail (fixed in scratch copy) |
| F-02 Confusion-matrix metrics | G-03 | 84 / 90 / 89.4 / 84.9 | Same | Pass |
| F-03 Exact AUC | G-07 | 0.9358 | 0.9294 | Fail |
| F-04 AUC from user grid | G-07 with [70, 75, 80] | Corners added, or refused | 0.08 | Fail |
| F-05 90 % CI | N-08 | z = 1.645 | z = 2.576 | Fail |
| F-06 Error codes | N-01 to N-03 | B002 / B002 / B001 | Same (thrown) | Pass |
| F-07 Unit check | N-05 | Reject | Accepted silently | Fail (engine); validator needed |
| F-08 String status | N-07 | Reject explicitly | All strings: B002; mixed: accepted and miscounted | Fail |
| F-09 CLI output | any | Derived from input | Fixed numbers | Fail |
A test suite that encodes these findings can be short. The sketch below uses Jest, which the package already configures; the helper codeOf returns the error code thrown by a call, so that assertions are about codes rather than message text.
import { BiomarkerDataSDK, BiomarkerErrorCode } from '@wia/bio-004';
const range = (a: number, b: number) =>
Array.from({ length: b - a + 1 }, (_, i) => a + i);
// Synthetic set S-100 (section 8.2.2)
const cases = [...range(76, 117), ...range(60, 67)];
const controls = [...range(30, 74), ...range(80, 84)];
const values = [...cases, ...controls];
const labels = [...cases.map(() => 1), ...controls.map(() => 0)];
const codeOf = (fn: () => unknown): string | undefined => {
try { fn(); return undefined; } catch (e: any) { return e.code; }
};
const sdk = new BiomarkerDataSDK();
const measurements = values.map((v, i) => ({
value: v, unit: 'ng/mL', diseaseStatus: labels[i] === 1,
}));
describe('golden vector G-03 / G-07', () => {
const res = sdk.validateBiomarker({
biomarkerType: 'protein',
measurements,
referenceRange: { min: 0, max: 75, unit: 'ng/mL' },
cutoffThreshold: 75,
prevalence: 0.5,
});
test('confusion matrix', () => {
expect(res.performance.confusionMatrix).toEqual({ tp: 42, tn: 45, fp: 5, fn: 8 });
});
test('percent scale', () => {
expect(res.performance.sensitivity).toBeCloseTo(84.0, 1);
expect(res.performance.npv).toBeCloseTo(84.9, 1);
});
test('operating point uses fractions', () => {
expect(res.rocAnalysis.optimalThreshold.sensitivity).toBeCloseTo(0.84, 4);
expect(res.rocAnalysis.optimalThreshold.youdenIndex).toBeCloseTo(0.74, 4);
});
test('exact AUC (fails on v1.0: 0.9294)', () => {
expect(res.rocAnalysis.auc).toBeCloseTo(0.9358, 3);
});
});
describe('negative vectors', () => {
test('N-01 length mismatch', () => {
expect(codeOf(() => sdk.calculateROC({ trueLabels: labels, predictions: values.slice(1) })))
.toBe(BiomarkerErrorCode.INVALID_MEASUREMENTS);
});
test('N-03 fewer than 100 samples', () => {
expect(codeOf(() => sdk.calculateROC({
trueLabels: labels.slice(0, 99), predictions: values.slice(0, 99),
}))).toBe(BiomarkerErrorCode.INSUFFICIENT_SAMPLES);
});
});Note that the test for F-03 is written to state the correct value and to fail on v1.0. A suite that encodes the current wrong value “so the build stays green” would turn a defect into a specification.
The phase annexes sketch, in non-normative terms, what a conformance claim should be delivered as: an evidence package and a profile manifest. The idea is sound and independent of the annexes’ boilerplate details, so we describe it generically.
The evidence package is a single archive that lets an assessor re-run the claim without contacting the vendor. It contains a software bill of materials (SBOM) in a recognised format such as SPDX or CycloneDX, so that the exact dependency versions are known; the API description of any network interface; the test vectors used, with their expected values; the pass/fail matrix produced by running them, including skipped tests and the reason for each skip; and a manifest listing every file with its hash, signed by the vendor. Negative results belong in the package with the same prominence as positive ones. A skipped test with no recorded reason should be read as a failure.
The profile manifest states which requirements are claimed. Each checklist row gets one of three statuses: supported, partial (with a reason) or excluded (with a justification). This avoids the most common way conformance claims mislead, which is by implying full coverage when only the easy parts were tested. The example below uses the checklist identifiers from §8.1.3 and the findings from §8.3; the field names are this book’s suggestion, not a schema defined by v1.0.
{
"standard": "WIA-BIO-004",
"standardVersion": "1.0.0",
"implementation": { "name": "example-biomarker-engine", "version": "2.3.1" },
"requirements": [
{ "id": "C-01", "status": "supported" },
{ "id": "C-03", "status": "partial",
"reason": "AUC uses a corrected trapezoid sort; CI method is Hanley-McNeil only" },
{ "id": "C-07", "status": "supported",
"reason": "Enforced by record validator ahead of statistics engine" },
{ "id": "C-12", "status": "excluded",
"justification": "Study-level requirement; outside software scope" }
],
"evidence": {
"sbom": "sbom.spdx.json",
"vectors": "vectors/",
"results": "results/matrix.json"
},
"signature": "<detached signature over the manifest hash>"
}Everything in this example is illustrative: the implementation name, version and statuses describe a hypothetical engine that has fixed F-03, not any real product.
Every WIA-BIO-004 record carries "version": "1.0.0", and the phase annexes commit the standard to Semantic Versioning 2.0.0: a major increment for backward-incompatible changes to normative requirements, a minor increment for new requirements that do not break existing implementations, and a patch increment for editorial changes. Applied to biomarker records, that rule has concrete consequences, summarised below. The examples are drawn from inconsistencies discussed in Chapter 3 and are hypothetical future changes, not announced ones.
| Change | Example | Increment |
|---|---|---|
| New optional field | Adding aucCIMethod to rocAnalysis | Minor |
| New value in an open vocabulary | Adding a sampleType such as dried blood spot | Minor |
| Renaming or retyping a field | Unifying min/max with lowerLimit/upperLimit | Major |
| Removing an enum value | Retiring diagnostic_validation in favour of the SDK’s studyType set | Major |
| Changing a default that alters results | Changing the PPV default from prevalence-based to count-based | Major |
| Correcting an example or typo | Fixing the §5.3.1 PPV/NPV example values | Patch |
Three version numbers are in play and must not be confused: the version of the standard a record conforms to (the record’s version field), the version of the SDK that produced a result (getVersion(), also copied into BiomarkerReport.metadata.version), and the version of the implementation making the claim. In v1.0 all three read “1.0.0”, which hides the distinction. A consumer should key its parsing on the record’s version, not on the SDK version, and a manifest should list all three.
§10.2 lists five regulatory and laboratory-standards documents. Conformance to WIA-BIO-004 is not compliance with any of them, and none of them certifies WIA-BIO-004. What the standard can do is produce records that make the evidence those documents ask for easier to assemble and to audit. The table corrects two details in §10.2: the EMA guideline has been replaced by ICH M10, and C28-A3 was published in 2008, with the 2010 date belonging to its corrected reprint C28-A3c.
| Document | Scope | WIA-BIO-004 evidence that feeds it | Caution |
|---|---|---|---|
| FDA, Bioanalytical Method Validation (2018) | Validation of methods measuring drugs and metabolites in study samples | Precision, accuracy, LOD/LOQ and stability records (Chapter 5) | Written for drug assays; biomarker assays call for a fit-for-purpose approach tied to the context of use |
| EMA, Guideline on bioanalytical method validation (2011) | As above, European Union | As above | Superseded by ICH M10 (2022), whose scope is drug and metabolite concentrations, not biomarkers |
| CLSI EP09c (2018) | Measurement procedure comparison and bias estimation with patient samples | Method-comparison data; measurementMethod and bias results | Needs raw paired results, not only summary statistics |
| CLSI EP15-A3 (2014) | User verification of precision and estimation of bias | Replicate data and CV records | A verification in the user’s laboratory, not a full validation |
| CLSI C28-A3 (2008; C28-A3c 2010) | Defining, establishing and verifying reference intervals | §3.3.2 reference-range records and their partitions | Record the number of reference individuals and the method (parametric or non-parametric) |
Two other frameworks are not in §10.2 but are what assessors usually ask about. For the diagnostic-accuracy study itself, the STARD reporting guideline lists what a publication must disclose: participant flow, the reference standard, blinding, and the handling of indeterminate results. Most of these map onto §7.2 fields, and the remainder, such as indeterminate results, have no field in v1.0. For the laboratory that runs the assay, ISO 15189 sets requirements for quality and competence, including internal QC and external quality assessment, the processes that §8.1’s “quality control” component and Chapter 5’s Westgard rules implement. The applicable in-vitro diagnostic regulation, for example the EU IVDR or the US CLIA framework, depends on jurisdiction and intended use, and is outside what any data standard can decide.
Writing this book surfaced a set of internal inconsistencies. None of them is hard to fix, and each makes conformance testing harder until it is fixed:
studyType has three incompatible value sets, and reference ranges use two field-name conventions. Chapter 3 recommends the SDK enum; the standard should say so.Beyond repairs, several developments will test the standard’s model of “one analyte, one value, one cutoff”.
Multi-omics composite scores. Many modern biomarkers are weighted combinations of proteins, transcripts and metabolites. The multi-analyte definition in Chapter 3 already has analytes[] and an algorithm block, but conformance testing needs more: the weights, the software version that computed the score, and a locked model so that the validated score is the one deployed. Chapter 6’s warnings about over-fitting apply with extra force.
Liquid biopsy and circulating tumour DNA. Results such as variant allele fraction from cell-free DNA sit between the genetic and quantitative worlds. They need the VCF-level provenance of Chapter 3 together with the pre-analytical controls of Chapter 5, since collection tubes and processing delay affect results, and their limits of detection are expressed differently from those of an immunoassay.
Digital biomarkers. Measures derived from wearables, smartphones or voice recordings are continuous streams processed by algorithms. The standard’s measurementMethod and laboratoryID fields do not describe a sensor, a firmware version or a feature-extraction pipeline, and analytical validation concepts such as LOD translate only partially.
Machine-learning-derived biomarkers. When the “value” is a model output, discrimination (AUC) is not enough. Calibration, meaning whether a predicted 30 % risk corresponds to an observed 30 % event rate, must be reported, together with the training population and a plan for monitoring drift. Reporting guidance for clinical prediction models, such as TRIPOD and its extensions for AI, is the natural companion here.
FHIR Genomics. Chapter 3 mapped quantitative results to FHIR Observation. For genetic biomarkers the HL7 FHIR Genomics Reporting implementation guide defines richer profiles for variants and their interpretation. Aligning WIA-BIO-004’s genetic records with those profiles would spare integrators a second mapping.
Each of these directions raises the same question this chapter started with: how would someone else check that it was done right? A standard that answers that question with shared vectors, disclosed methods and honest manifests will be useful long after the particular numbers in v1.0 have been revised.
Conformance to WIA-BIO-004 v1.0 is assembled from four sources: the five required components, the data records, the error codes and the seven validation requirements. The first three can be tested mechanically, and the arithmetic core of the standard makes exact test vectors possible. Running such vectors against the reference SDK confirms its confusion-matrix arithmetic and error codes, and exposes defects in ROC integration, partial threshold grids, confidence-level handling and unit checking, as well as a build error and a CLI that does not yet read data. An evidence package with a signed profile manifest turns those results into a claim that others can verify. Semantic versioning, alignment with the listed regulatory and laboratory guidances, and a short list of housekeeping repairs complete the path to the next revision. Future biomarkers, whether composite, circulating, digital or learned, will stretch the standard’s data model, but the discipline of checkable claims carries over unchanged.
B003 in terms of a §7.2 record, and explain why it cannot be tested against the v1.0 SDK directly.aucCIMethod; renaming referenceRange.min; correcting the §5.3.1 example.