Chapter 6
An assay that measures an analyte precisely has passed only the first test. This chapter covers the second and third: showing that the measurement separates patients who have a condition from those who do not, predicts how they will fare, tracks their response to treatment, and adds something to what clinicians already know.
Chapter 5 dealt with analytical validation: whether the assay measures the analyte correctly, reproducibly and stably. Clinical validation asks a different question. Given a measurement we trust, does it carry information about the patient's clinical state? A troponin assay with an excellent coefficient of variation is still useless as a diagnostic test if the chosen cutoff misses most infarctions, and a prognostic score with a striking hazard ratio in one hospital may collapse in the next because the first study selected its patients in a way that flattered the marker. WIA-BIO-004 addresses this stage in several places: the statistical methods of §4 and §5, the clinical-correlation methods of §6, the validation-study record of §7.2, the "Clinical Validation Metrics" and "Clinical Utility Metrics" sections of the quality-metrics phase document, the seven validation requirements in the README, and the SDK types ClinicalCorrelation, PrognosticAnalysis, TreatmentResponse and ValidationStudy.
The chapter follows the order in which a real validation programme unfolds: design the study, size it, validate the model internally and externally, then examine the marker's relationship to disease stage, outcome and treatment, and finally ask whether it improves decisions. Throughout, the statistics of Chapter 2 (confusion matrix, ROC-AUC, likelihood ratios) are taken as known; here they are applied to a study population rather than to a table of counts.
The quickest way to estimate sensitivity and specificity is a case-control (two-gate) design: recruit patients whose diagnosis is already confirmed, recruit a separate group of controls, measure the biomarker in both and build the confusion matrix. The design is efficient, which is why the discovery phase almost always starts here. Its weakness is that the two groups are chosen at the extremes. Cases tend to be advanced, symptomatic and well documented; controls tend to be healthy volunteers. The biomarker is asked to separate people who differ in almost every respect, and it usually succeeds better than it will in practice.
A prospective cohort (single-gate) design enrols consecutive patients in whom the clinical question actually arises, such as everyone presenting with chest pain or everyone referred for a suspicious imaging finding, measures the biomarker at entry and applies the reference standard to all of them. The resulting spectrum of disease severity and of look-alike conditions matches the intended use, so the accuracy estimates transfer to practice. The cost is size and time: if the condition is present in a fifth of those enrolled, four non-cases must be recruited for every case.
Spectrum bias is the distortion just described: accuracy depends on who is tested. A marker evaluated in metastatic cancer against young healthy donors will show a higher AUC than the same marker evaluated in early-stage disease against patients with benign conditions that raise the same analyte. The WIA-BIO-004 validation-study record makes spectrum visible by requiring a population block with age range, sex distribution and ethnicity; implementers should also record disease stage and the source of the controls, for which PatientSample.diseaseStage and medicalHistory provide places in the SDK.
Verification bias occurs when only some participants, typically those with a positive biomarker result, receive the reference standard. Sensitivity is then overestimated because missed cases among biomarker-negative patients are never found. Incorporation bias arises when the biomarker itself contributes to the reference diagnosis. Review bias arises when whoever interprets the reference standard knows the biomarker result, or vice versa.
The README's fourth requirement addresses the last of these directly: measurements must be performed blind to clinical status. In a data pipeline this translates into a concrete control. The laboratory system that produces the biomarker record should receive a coded sample identifier with no diseaseStatus, and the linkage to outcome should happen only after the measurement is locked. The §7.1 measurement record supports this, because clinicalContext.diseaseStatus may be null at the time of measurement and populated later.
Every accuracy estimate is relative to a reference standard: histopathology for many cancers, angiography or a universal definition of infarction for cardiac injury, culture or sequencing for infection. The §7.2 record stores it in referenceStandard with a method name and the standard's own sensitivity and specificity, conventionally 100/100. Setting both to 100 is an assumption, not a measurement; when the reference is known to be imperfect (for example a clinical adjudication panel) the record should say so in the SDK field referenceStandard.description, because an imperfect reference biases the biomarker's apparent accuracy, usually downward.
WIA-BIO-004 §4.3.2 names three stages with minimum recommended sizes, and the SDK's ValidationStudy.studyType enumerates the same progression plus a multi-centre variant. The quality-metrics phase document gives larger ranges for the later stages. The two sources are reproduced side by side below; neither is a statistical power calculation, which is the subject of section 6.2.
| Stage | SDK studyType | §4.3.2 minimum | Quality-metrics document | Main question |
|---|---|---|---|---|
| Discovery | discovery | 50 cases, 50 controls | not stated | Is there any signal worth pursuing? |
| Clinical validation | validation, multi_center | 100 cases, 100 controls | n = 200–500+ | How accurate is a fixed test in the intended population? |
| Clinical utility | clinical_utility | 200+ cases, 200+ controls | n = 500–1000+ | Does using the test change management and outcomes? |
Two further numbers must be kept apart. The README requires "a minimum of 100 cases and 100 controls" for initial validation, while the SDK constant MIN_VALIDATION_SAMPLES = 100 checks the total number of measurements. A dataset of 70 cases and 30 controls therefore passes the SDK check yet fails the README requirement. A conformant pipeline should test both, and report the per-group counts in the population block.
The following checklist combines the README requirements with established reporting practice. For diagnostic accuracy studies the STARD statement is the usual reporting guideline and QUADAS-2 the usual tool for appraising risk of bias; for prognostic tumour markers the REMARK guideline plays the corresponding role, and for multivariable prediction models TRIPOD. The standard does not mandate any of them, but a validation report that follows one is far easier to audit.
| Item | What to decide | Where it is recorded |
|---|---|---|
| Intended use | Diagnostic, prognostic, predictive or monitoring role; target population; the clinical decision affected | Biomarker definition (indication), study protocol |
| Design | Case-control or consecutive cohort; single or multi-centre | studyType |
| Spectrum | Disease stages included; source and type of controls, including look-alike conditions | population, PatientSample.diseaseStage |
| Reference standard | Method, timing relative to sampling, applied to all participants or not | referenceStandard |
| Blinding | Laboratory blind to clinical status; reference readers blind to biomarker (README requirement 4) | Protocol; diseaseStatus populated only after measurement lock |
| Cutoff | Pre-specified value or rule for choosing it (Chapter 2); never tuned in the validation set | rocAnalysis.optimalThreshold |
| Sample size | Target precision or power, per-group counts (README requirements 1–2) | population.cases, controls |
| Independent cohorts | At least two independent populations (README requirement 3) | validation.validationCohorts |
| Pre-analytics | Sampling and storage conditions controlled and documented (README requirement 7, Chapter 5) | qualityFlags, sampleType |
| Missing and indeterminate results | How excluded, flagged or imputed samples are counted | Report narrative; per-sample flags |
When the goal is to estimate sensitivity (or specificity) with a confidence interval of known width, the quality-metrics phase document gives the familiar normal-approximation formula:
n = Z² × p × (1 − p) / d²
Z = 1.96 (two-sided 95 % confidence)
p = expected sensitivity (or specificity)
d = desired half-width of the confidence interval
Example: p = 0.90, d = 0.05
n = 3.8416 × 0.90 × 0.10 / 0.0025 = 138.3 → 139 diseased participantsThe phase document quotes the result as 138; since sample sizes are rounded up, 139 is the strict answer, although the difference is immaterial. The same calculation applies separately to specificity, using the number of non-diseased participants. Two points are often overlooked. First, p enters as p(1 − p), so a lower expected value needs more participants: at p = 0.84 and d = 0.05 the requirement rises to about 207. Second, the result is the number of cases. In a prospective cohort where the condition is present in a proportion π of those enrolled, the total enrolment is n / π; with π = 0.20 (an illustrative value), 139 cases require about 695 participants.
For a continuous marker the quantity of interest is usually the AUC. The Hanley and McNeil standard error, introduced in Chapter 2 and listed in §10.3 of the specification, can be used in reverse: choose an expected AUC and group sizes, compute the standard error, and increase the groups until 1.96 × SE reaches the desired half-width. With equal groups and an expected AUC of 0.85, the half-width is about 0.082 with 44 per group, about 0.054 with 100 per group, and falls below 0.05 at roughly 117 per group. Obuchowski's methods extend this idea to comparisons of two AUCs and to clustered designs; dedicated software implementing them is widely available.
About the formula in §4.3.2 and Appendix A.2. The specification gives n = (Zα + Zβ)² · AUC(1 − AUC) / [(AUC − 0.5)² · prevalence · (1 − prevalence)] and, in Appendix A.2, derives "44 cases and 44 controls" for an expected AUC of 0.85 (the arithmetic actually gives about 43.7). This expression is not one of the established sample-size methods for ROC studies, and 44 per group is not an adequate design target: as shown above, it yields a confidence interval of roughly ±0.08 around the AUC. This book therefore recommends sizing by precision (6.2.1) or by the Hanley–McNeil or Obuchowski approach (6.2.2), and treating the §4.3.2 stage minimums as floors rather than results.
The README's second requirement asks for at least 80 % power to detect a clinically meaningful difference. That is only meaningful once the difference has been named, for example "the new marker's AUC exceeds the current marker's by at least 0.05". Sizing then follows the correlated-AUC methods above, because both markers are measured in the same people. Whatever method is used, the planned numbers must be inflated for losses: for a sample of n analysable participants and an expected fraction r of dropout, haemolysed or otherwise unusable samples, enrol n / (1 − r). With n = 139 and r = 0.10, that is 155. Samples excluded for quality reasons should keep their record and flag rather than disappear, so that the loss rate itself can be audited.
When the same data are used to choose a cutoff, select analytes or fit weights and then to measure accuracy, the result is biased upward. The bias is small for a single marker with a pre-specified cutoff and large for a multi-marker score. The quality-metrics document defines such a composite as Score = Σ(wᵢ × xᵢ), a weighted sum of normalised values. With twenty candidate analytes and a hundred participants, some weighting will separate cases from controls almost perfectly by chance, and that separation will not reappear in new patients.
The phase document lists k-fold cross-validation (typically k = 5 or 10), leave-one-out cross-validation (LOOCV) for small datasets, and an independent validation cohort as the reference approach. In k-fold cross-validation the data are split into k parts; the model is built on k − 1 parts and evaluated on the remaining one, and the k estimates are averaged. The essential rule is that every data-driven step, including analyte selection, normalisation parameters, weight fitting and cutoff choice, must be repeated inside each training fold. Selecting analytes on the full dataset and cross-validating only the weights reproduces the optimism that cross-validation was meant to remove. Repeating k-fold with different random splits reduces the variance of the estimate; the SDK's ValidationStudy.crossValidation object has fields for this: method, folds, iterations, averageAUC and aucStdDev.
LOOCV is a special case with k equal to the sample size. It uses the data efficiently but its estimate can be highly variable, and it is not a substitute for an external cohort. The §7.2 record stores the chosen scheme as a string such as "10-fold".
External validation applies the frozen model, meaning the same analytes, weights and cutoff, to participants from a different site, period or population. README requirement 3 asks for at least two independent populations. A drop in AUC between development and external cohorts is expected and informative; a large drop suggests overfitting or a spectrum difference. The §7.2 record lists each external cohort with its size and AUC, and the SDK adds sensitivity and specificity at the frozen cutoff, which are the numbers a clinical user actually experiences.
Subgroup analysis asks whether performance is consistent across age, sex, ethnicity, disease stage or comorbidity. The §7.2 example splits its population at age 60. Subgroup AUCs are estimated with less precision than the overall AUC and should be reported with confidence intervals; differences between subgroups should be tested formally rather than read off point estimates. A pre-specified short list of subgroups is more credible than a search through many.
When a multi-marker model outputs a probability (as the SDK's predictOutcome does through its logistic model, Chapter 4), discrimination is not enough. The predicted probabilities must also agree with observed event rates. Calibration is usually summarised by a calibration plot, a calibration slope (ideally 1) and a calibration-in-the-large intercept (ideally 0) in the external cohort. WIA-BIO-004 v1.0 has no fields for calibration; until it does, calibration results belong in the report narrative, and Chapter 8 lists this as an open item.
Many markers are expected to rise, or fall, with disease severity. Section 6.1 of the specification names three analyses for this relationship. Its example uses four stages with illustrative means of 50, 75, 120 and 180 units and a Spearman correlation of 0.85; these values show the shape of a report and are not results from a real study.
| Method | Question answered | Assumptions and notes |
|---|---|---|
| Spearman rank correlation (ρ) | Is the marker monotonically associated with stage? | Rank-based, so suitable for ordinal stage and skewed marker values; many tied stage ranks reduce its information |
| Kruskal–Wallis test | Does the marker distribution differ between any of the stages? | Ignores the ordering of stages; a significant result does not show a trend |
| Trend test (for example Jonckheere–Terpstra, or linear regression on stage score) | Is there an ordered increase or decrease across stages? | Uses the ordering; more powerful than Kruskal–Wallis when a monotone trend is expected |
Biomarker concentrations are frequently right-skewed, so reporting medians and interquartile ranges per stage is more informative than means and standard deviations. If a parametric model is preferred, a logarithmic transformation usually brings the residuals closer to normal. Statistical significance alone says little: with a few hundred participants even a weak correlation will be "significant", so the coefficient and its confidence interval matter more than the p-value. Overlap between adjacent stages is also clinically important, because a marker whose stage II and stage III distributions overlap heavily cannot be used to assign stage to an individual even if the group trend is strong.
The SDK records such an analysis as a ClinicalCorrelation. Its outcome is a ClinicalOutcome whose type is one of survival, progression, response, recurrence, severity or other, and whose measurement scale is continuous, binary, ordinal or time-to-event. A stage correlation might be stored as follows (values illustrative):
{
"id": "CORR-STAGE-01",
"biomarkerID": "BIO-67890",
"outcome": {
"type": "severity",
"name": "Tumour stage (I-IV)",
"measurement": "ordinal"
},
"correlationType": "spearman",
"correlationCoefficient": 0.62,
"pValue": 0.00001,
"sampleSize": 180,
"isSignificant": true
}
The type allows pearson, spearman and kendall; there is no field for the confidence interval of the coefficient or for the group-comparison tests, so these go into the accompanying report. The isSignificant flag should be derived from the pre-specified α (the SDK constant ALPHA is 0.05) rather than set by hand.
A prognostic biomarker predicts the course of disease, such as death, recurrence or progression, independent of treatment. Because follow-up times differ and some participants have not yet had the event when the study closes (they are censored), the appropriate tools are survival methods. Section 6.2.1 of the specification uses the Cox proportional hazards model:
h(t | x) = h₀(t) × exp(β × x)
h(t | x) hazard at time t for a patient with marker value x
h₀(t) baseline hazard, left unspecified
β log hazard ratio per unit of x
HR = exp(β)
HR > 1 higher marker, higher risk (poor prognosis)
HR < 1 higher marker, lower risk
HR = 1 no associationThe hazard ratio depends entirely on the unit of x. If β = 0.0069 per ng/mL (an illustrative value), the HR per 1 ng/mL is 1.007, which looks negligible, while the HR per 100 ng/mL is exp(0.69) ≈ 2.0. A report should therefore state the scale explicitly: per unit, per standard deviation, per doubling (using log₂ of the concentration) or for high versus low around a pre-specified cutoff. Dichotomising at the median or at an "optimal" cutpoint chosen by searching over survival p-values discards information and inflates the apparent effect; if a cutoff is needed for clinical use, it should be fixed before the survival analysis.
Two further checks belong in any Cox analysis. The proportional-hazards assumption (that the HR is constant over time) should be examined, for example with scaled Schoenfeld residuals; markers measured at diagnosis often lose predictive strength over years of follow-up. And the marker's value should be shown to persist after adjustment for established prognostic factors such as stage, age and performance status. A marker that merely mirrors stage adds nothing.
The hazard ratio measures association, not discrimination. Harrell's concordance index (C-index) is the survival analogue of the AUC: among all usable pairs of patients, meaning pairs where the one with the shorter follow-up actually had the event, it is the proportion in which the patient with the higher predicted risk had the event first. A C-index of 0.5 is chance and 1.0 perfect ordering.
Section 6.2.2 of the specification defines a time-dependent AUC and asks for it at 1, 3 and 5 years. In the common "cumulative/dynamic" form, cases at time t are patients who had the event by t, and controls are those still event-free at t; AUC(t) is the probability that a random case has a higher marker value than a random control. Because censored patients have unknown status at t, a naive calculation that drops them is biased; estimators that weight observations by the inverse probability of censoring, or the nearest-neighbour approach of Heagerty and colleagues, handle this properly. Reporting AUC(t) at several horizons shows whether the marker's prognostic information fades.
The SDK's PrognosticAnalysis type carries the hazard ratio with its 95 % confidence interval, the p-value, an optional C-index, optional survival curves for high and low marker groups (arrays of survival probabilities at listed time points) and optional median survival with a time unit. The values below are illustrative:
{
"biomarkerID": "BIO-67890",
"hazardRatio": 2.0,
"hazardRatioCI95": [1.4, 2.9],
"pValue": 0.0002,
"cIndex": 0.68,
"survivalCurves": {
"timePoints": [12, 36, 60],
"high": [0.82, 0.61, 0.47],
"low": [0.93, 0.83, 0.74]
},
"medianSurvival": { "high": 54, "low": 96, "unit": "months" }
}The type has no field for the unit of the hazard ratio, the adjustment covariates or AUC(t). Until those exist, include them in the report text and name the covariates explicitly. Note also that a median survival cannot be estimated when fewer than half of a group have had the event; in that case omit the field rather than extrapolate.
For monitoring biomarkers, section 6.3.1 of the specification classifies response by the change from a pre-treatment baseline. These categories are the standard's own definitions; they are not the imaging-based RECIST criteria, nor any disease-specific consensus such as the prostate cancer working group criteria for PSA, and they should be described as WIA-BIO-004 categories in any report.
| Category | SDK value | WIA-BIO-004 definition |
|---|---|---|
| Complete response (CR) | complete | Marker within the normal (reference) range |
| Partial response (PR) | partial | Decrease of 50 % or more from baseline |
| Stable disease (SD) | stable | Decrease of less than 50 % and increase of less than 25 % |
| Progressive disease (PD) | progressive | Increase of 25 % or more from baseline |
Percent change is (value − baseline) / baseline × 100, stored in TreatmentResponse.percentChange. For example, a baseline of 200 ng/mL falling to 80 ng/mL is a change of −60 %, a partial response unless 80 ng/mL lies inside the reference range, in which case it is complete. An implementation needs precedence rules that v1.0 leaves implicit: test CR first, then PR, PD and SD; define the baseline window (the last value before treatment starts); and decide whether a single measurement suffices or a confirmatory sample is required. Because analytical imprecision alone can move a value by several percent, a change near the 25 % boundary should be interpreted against the assay's total CV from Chapter 5, and ideally against its reference change value. A baseline inside the reference range makes CR meaningless, so such patients should be flagged rather than classified.
When a marker falls after effective therapy, the rate of decline carries information in its own right. If the decline is approximately first-order, the value follows C(t) = C₀ · e−kt, and the specification relates the rate constant to the half-life:
k = ln 2 / T½ (equivalently −ln 0.5 / T½)
Example (illustrative): T½ = 3 days
k = 0.693 / 3 = 0.231 per day
after 9 days (three half-lives) the value is 1/8 of baseline
In practice k is estimated by linear regression of ln(value) on time over the early post-treatment samples, which requires at least three well-timed measurements above the limit of quantification. Values that reach the LOQ must be handled as censored, not as exact numbers, or the fitted half-life will be too short. TreatmentResponse stores the raw series in longitudinalData (time point, value and unit), plus optional timeToResponse and durationOfResponse; the fitted half-life has no dedicated field.
Serial measurements from the same patient are correlated, so ordinary regression that treats every sample as independent understates uncertainty. Section 6.3.2 of the specification uses a linear mixed-effects model:
Biomarker_ij = β₀ + β₁ × Time_ij + β₂ × Treatment_i
+ β₃ × (Time × Treatment)_ij + u_i + ε_ij
i subject j time point
u_i random subject effect, u_i ~ N(0, σ²_u)
ε_ij residual error, ε_ij ~ N(0, σ²)
β₃ difference in slope between treatment groupsβ₁ is the slope over time in the reference group, β₂ the baseline difference between groups and β₃, the coefficient of interest, the treatment effect on the trajectory. The random intercept ui lets each patient have their own level. A random slope for time is a natural extension when patients differ in how fast the marker changes. Mixed models accommodate unequal numbers and irregular timing of samples, which is typical of routine monitoring, but they assume that missing samples are missing at random given the observed data. If patients who progress stop being sampled, that assumption fails and a joint model of the marker and the event may be needed. Analysis on the log scale is usually preferable for concentrations, in which case β₃ is interpreted as a ratio of rates of change.
A marker can be accurate, prognostic and responsive to treatment yet still be clinically useless if it tells clinicians nothing they did not already know from age, symptoms and existing tests. The final stage of validation therefore asks two questions: does the marker improve on the existing model, and does acting on it lead to better decisions?
NRI compares risk categories assigned by an old model and by the new model that adds the biomarker. Section 5.2.1 of the specification defines it as the sum of two net proportions:
NRI = [P(up | event) − P(down | event)] + [P(down | non-event) − P(up | non-event)]
Example (illustrative), risk categories low / intermediate / high
100 events: 20 moved up, 8 moved down → (20 − 8) / 100 = 0.12
400 non-events: 60 moved down, 28 moved up → (60 − 28) / 400 = 0.08
NRI = 0.12 + 0.08 = 0.20NRI should always be reported as its two components, because the same total can hide opposite effects: a model that moves everyone up gains among events and loses among non-events. NRI depends on the choice and number of categories, which should be clinically motivated and fixed in advance. The category-free version, which counts any upward or downward movement of predicted risk, is known to report positive values even for uninformative markers and should be interpreted cautiously.
IDI avoids categories. The discrimination slope of a model is the mean predicted probability among events minus the mean predicted probability among non-events. IDI is the new model's discrimination slope minus the old model's. The specification's §5.2.2 labels the terms "ΔSensitivity", which is misleading; the correct quantities are mean predicted probabilities:
IDI = [mean p_new(events) − mean p_new(non-events)]
− [mean p_old(events) − mean p_old(non-events)]
Example (illustrative)
new model: 0.42 − 0.14 = 0.28
old model: 0.35 − 0.16 = 0.19
IDI = 0.28 − 0.19 = 0.09IDI rewards models that push predicted risks further apart, which can also happen if the new model is poorly calibrated. It is therefore reported together with calibration measures and the change in AUC or C-index, not as a stand-alone proof of value.
Discrimination measures treat false positives and false negatives symmetrically, but clinical decisions do not. Decision curve analysis, given in the quality-metrics document, weights false positives by the odds of the threshold probability pt at which a clinician would act:
Net Benefit = TP / n − (FP / n) × p_t / (1 − p_t)
Example (illustrative): n = 1000, 200 events, p_t = 0.20 → weight 0.25
Biomarker-guided: TP = 160, FP = 120 → 0.160 − 0.120 × 0.25 = 0.130
Treat all: TP = 200, FP = 800 → 0.200 − 0.800 × 0.25 = 0.000
Treat none: → 0The threshold probability encodes a value judgement. At pt = 0.20 a clinician accepts four unnecessary interventions for each correctly treated patient. Net benefit is expressed in "true positives per patient", so in the example the biomarker strategy is equivalent to finding 130 true cases per 1000 patients with no false positives. A decision curve plots net benefit across a range of reasonable thresholds for the biomarker strategy, for treat-all and for treat-none; a marker is useful where its curve lies above both. Unlike AUC, net benefit requires realistic prevalence, so it must be computed in a cohort that reflects the intended-use population, not in a case-control sample.
The quality-metrics document also defines NNT = 1 / |Outcomebiomarker − Outcomestandard|, the number of patients who must be managed with the biomarker strategy rather than the standard strategy to change one outcome. If an adverse outcome occurs in 12 % under standard care and 8 % under biomarker-guided care (illustrative figures), NNT = 1 / 0.04 = 25. The outcome rates must come from a comparative study, ideally randomised, in which management actually depended on the biomarker. Accuracy data alone cannot produce an NNT, which is why the clinical-utility stage in 6.1.4 needs the largest samples of all.
The §7.2 validation-study record brings the chapter together. The example below follows the specification's structure with its numbers checked. The 300 participants split 150 cases and 150 controls, 145 men and 155 women, and 180/60/40/20 across the four ethnicity groups. At the reported cutoff, sensitivity 84 % and specificity 90 % correspond to TP 126, FN 24, TN 135 and FP 15, which give PPV 126/141 = 89.4 %, NPV 135/159 = 84.9 %, accuracy 261/300 = 87.0 %, LR+ = 0.84/0.10 = 8.4, LR− = 0.16/0.90 = 0.18 and Youden J = 0.74. The confidence intervals have been recomputed: the Hanley–McNeil interval for AUC 0.87 with 150 per group is about 0.83–0.91, and Wilson intervals are 77.3–89.0 % for sensitivity and 84.2–93.8 % for specificity. In line with Chapter 3, studyType uses the SDK value validation instead of the specification's diagnostic_validation. The biomarker name and all results are illustrative.
{
"standard": "WIA-BIO-004",
"version": "1.0.0",
"studyID": "VALID-2025-001",
"studyType": "validation",
"biomarker": {
"id": "BIO-67890",
"name": "Novel Cancer Marker",
"type": "protein"
},
"population": {
"totalParticipants": 300,
"cases": 150,
"controls": 150,
"ageRange": [45, 75],
"sexDistribution": { "male": 145, "female": 155 },
"ethnicity": { "caucasian": 180, "african": 60, "asian": 40, "hispanic": 20 }
},
"referenceStandard": {
"method": "Histopathological confirmation",
"sensitivity": 100,
"specificity": 100
},
"results": {
"rocAnalysis": {
"auc": 0.87,
"aucCI95": [0.83, 0.91],
"optimalThreshold": {
"value": 125.5,
"unit": "ng/mL",
"sensitivity": 84.0,
"specificity": 90.0,
"youdenIndex": 0.74
}
},
"performanceMetrics": {
"sensitivity": 84.0,
"sensitivityCI95": [77.3, 89.0],
"specificity": 90.0,
"specificityCI95": [84.2, 93.8],
"ppv": 89.4,
"npv": 84.9,
"accuracy": 87.0,
"lrPositive": 8.4,
"lrNegative": 0.18
},
"subgroupAnalysis": [
{ "subgroup": "Age <60", "n": 140, "auc": 0.89 },
{ "subgroup": "Age >=60", "n": 160, "auc": 0.85 }
]
},
"validation": {
"crossValidation": "10-fold",
"externalValidation": true,
"validationCohorts": [
{ "cohort": "External Cohort 1", "n": 200, "auc": 0.84 }
]
}
}
Three cautions apply when reading such a record. The PPV and NPV above hold only at the study's 50 % case fraction; for any other prevalence they must be recomputed with the Bayes form from Chapter 2, which is what the SDK's validateBiomarker does with its prevalence parameter. Sensitivity and specificity are in percent here, as in the specification and in the SDK's DiagnosticPerformance, whereas SDK ROC points use fractions from 0 to 1 (Chapter 4). And the record reports one external cohort, while README requirement 3 asks for at least two independent populations, so this study on its own does not complete the validation stage.