Chapter 2
How many subjects a trial needs, how likely it is to detect a real effect, and how the final and interim results are tested — worked by hand, checked in code, and compared line by line with §4 and Appendix A of WIA-BIO-010 v1.0.
@wia/bio-010 SDK's statistics functions against the hand calculations so that you know which outputs you can trust.
A confirmatory trial is a pre-registered attempt to reject a null hypothesis. The protocol commits in advance to the primary endpoint, the population in which it will be analysed, the test and the significance level. The data are then allowed to answer one question. The design task in this chapter comes down to choosing a number of subjects large enough that the question gets a clear answer, and small enough that no more people are exposed to an unproven treatment than necessary.
Spec §4.3 writes the primary comparison of two arms with means μ₁ (experimental) and μ₂ (control) in three forms:
H0: mu1 = mu2 (no difference)
H1: mu1 != mu2 (two-sided superiority or difference)
H1: mu1 > mu2 (one-sided superiority)
H1: mu1 - mu2 > -delta (non-inferiority, margin delta > 0)For non-inferiority, the null hypothesis is the complement: H₀: μ₁ − μ₂ ≤ −δ, meaning the new treatment is worse than control by at least the margin δ. The margin is a clinical and regulatory judgement, usually derived from how large the control treatment's effect over placebo is known to be. ICH E10 on the choice of control group discusses this. It is not a statistical by-product. A trial that "fails to show a difference" has not shown non-inferiority. Only a trial designed around a margin, with a confidence interval whose lower bound stays above −δ, can support that claim.
Every test can be wrong in two ways. The protocol fixes the probability of each before any data are seen.
| H₀ true (no real effect) | H₁ true (real effect of size Δ) | |
|---|---|---|
| Reject H₀ | Type I error, probability α (false-positive claim of efficacy) | Correct, probability 1 − β = power |
| Do not reject H₀ | Correct, probability 1 − α | Type II error, probability β (an effective treatment abandoned) |
Regulators care most about α, because a Type I error puts an ineffective drug on the market. Sponsors and patients also care about β, because an underpowered trial wastes the participation of every enrolled subject. Spec §4.2 lists the four levers that move power: larger n, larger Δ and larger α raise it, and larger σ lowers it. Every formula below is one of these four quantities written in terms of the other three.
For a superiority claim, regulators generally expect a two-sided test at α = 0.05, or equivalently a one-sided test at α = 0.025. The two use the same critical value, 1.96. A one-sided test at 0.05 (critical value 1.645) reduces the sample size by about 20% and is generally not accepted as confirmatory evidence. The SDK exposes this choice through the testType field ('one-tailed' | 'two-tailed'). The SDK maps 'one-tailed' to 1.645 and 'two-tailed' to 1.96 whatever alpha you pass (see section 2.8).
Since the ICH E9(R1) addendum, the thing being estimated, the estimand, has to be defined before the test. It specifies the population, the variable, how intercurrent events such as rescue medication or treatment discontinuation are handled, and the summary measure (difference in means, risk ratio, hazard ratio). A sample-size formula is always written for one summary measure. If the estimand changes, for example from "while on treatment" to "treatment policy", the variance and the expected effect change with it, and the sample size must be recalculated. Spec §1.4 still lists LOCF (last observation carried forward) as a handling method. Chapter 1 explains why modern guidance no longer treats it as a default.
For a two-arm, 1:1 randomized comparison of means with common standard deviation σ, spec §4.1 gives the standard normal-approximation formula for the number of subjects per group:
n = (Z[1-alpha/2] + Z[1-beta])^2 * 2 * sigma^2 / Delta^2Each term has a clear role. The factor 2σ²/n is the variance of a difference between two independent means. The bracketed sum is the distance, in standard errors, that the true difference Δ has to sit from zero so that the test statistic crosses the critical value with probability 1 − β. The critical values are properties of the standard normal distribution:
| Quantity | Value | Z |
|---|---|---|
| α = 0.05 two-sided (or 0.025 one-sided) | Z0.975 | 1.960 |
| α = 0.01 two-sided (or 0.005 one-sided) | Z0.995 | 2.576 |
| α = 0.05 one-sided | Z0.95 | 1.645 |
| Power 80% (β = 0.20) | Z0.80 | 0.842 |
| Power 90% (β = 0.10) | Z0.90 | 1.282 |
| Power 95% (β = 0.05) | Z0.95 | 1.645 |
Squaring the sum gives the design multiplier, which is worth memorising because it shows the cost of stricter settings directly. It is 7.85 for (α 0.05, 80%), 10.51 for (α 0.05, 90%), 11.68 for (α 0.01, 80%) and 14.88 for (α 0.01, 90%). Moving from 80% to 90% power therefore costs about 34% more subjects. Moving from α 0.05 to 0.01 at 80% power costs about 49% more. The spec and the SDK use the rounded pair 1.96 and 0.84, whose square is 7.84. At the precision of a protocol, the difference from 7.85 almost never changes the rounded answer.
Appendix A.1 sizes a glucose-lowering trial whose primary endpoint is change in HbA1c. The inputs are α = 0.05 two-sided, 80% power, σ = 1.2 percentage points and a clinically meaningful difference Δ = 0.5 percentage points.
(1.96 + 0.84)^2 = 2.80^2 = 7.84
2 * sigma^2 = 2 * 1.2^2 = 2.88
Delta^2 = 0.5^2 = 0.25
n per group = 7.84 * 2.88 / 0.25 = 90.32
round UP per group -> 91
total (1:1) -> 182The rounding rule matters. Round the per-group figure up to the next whole subject, then double it. If you round the total, or round to the nearest integer, the design can fall just short of its stated power. Appendix A.1 gets this right at the per-group step (90.3 → 91 is implied). Its dropout step does not.
If a proportion d of randomized subjects is expected to provide no usable primary-endpoint data, the number to randomize is N / (1 − d), not N × (1 + d). With d = 0.20: 91 / 0.8 = 113.75, which rounds up to 114 per group, 228 in total. Appendix A.1 reports 113 per group and 226 in total because it inflates the unrounded 90.3. That is 3 subjects short of the conservative figure. The shortcut 91 × 1.2 = 109.2 is worse still: even rounded up to 110, it leaves only 88 evaluable subjects per group after 20% loss. Under an intention-to-treat estimand, dropouts are usually still analysed with imputed or modelled values rather than simply lost. Simple inflation is therefore a planning approximation, and the statistical analysis plan must say how missing data will actually be handled.
Spec §4.1 uses σ = 1.5 and Δ = 0.5 with the same α and power: 7.84 × 4.5 / 0.25 = 141.12. The text rounds this to 141 per group, 282 in total and 353 after dropout. Rounding 141.12 up gives 142 per group, 284 in total, and 284 / 0.8 = 355 to randomize. Use the corrected figures in any test vector or protocol template derived from the spec.
When a sponsor randomizes k experimental subjects for every control (for example to gain more safety exposure), the control-arm size becomes nC = (1 + 1/k) · σ² · (Z1−α/2 + Z1−β)² / Δ², and nT = k · nC. For the A.1 inputs at 2:1: nC = 1.5 × 1.44 × 7.84 / 0.25 = 67.7 → 68, nT = 136, total 204. That is 12% more than the 182 needed with 1:1 allocation for the same power. Balanced allocation is the most efficient when variances are equal.
For a non-inferiority design with margin δ, and assuming the true difference is zero, the same formula applies with Δ replaced by δ and a one-sided α of 0.025 (critical value 1.96). As an illustration only, a margin of 0.35 HbA1c percentage points with σ = 1.2 gives 7.84 × 2.88 / 0.1225 = 184.3 → 185 per group. Margins are usually smaller than the superiority effect one hopes for, so non-inferiority trials are typically larger.
When the primary endpoint is a yes/no event such as response, relapse or a cardiovascular event, the variance depends on the event rates themselves. Spec §4.1 gives the two-proportion formula, with p₁ and p₂ the expected event rates and p̄ = (p₁ + p₂)/2:
n = [ Z[1-alpha/2] * sqrt(2 * pbar * (1 - pbar))
+ Z[1-beta] * sqrt(p1 * (1 - p1) + p2 * (1 - p2)) ]^2 / (p1 - p2)^2The first square root is the standard deviation of the difference under H₀, where both arms share the pooled rate. The second is the standard deviation under H₁. The two terms are added before squaring. This detail matters again when we look at the SDK.
Suppose, for illustration, that the control arm is expected to have a 25% event rate over the treatment period and the experimental arm 15%, a 10-point absolute reduction. With α = 0.05 two-sided and 80% power:
p1 = 0.25, p2 = 0.15, pbar = 0.20
2 * pbar * (1 - pbar) = 0.320 sqrt = 0.5657
p1(1-p1) + p2(1-p2) = 0.1875 + 0.1275 = 0.315 sqrt = 0.5612
term A = 1.96 * 0.5657 = 1.1087
term B = 0.84 * 0.5612 = 0.4714
(A + B)^2 = 1.5802^2 = 2.4970
(p1 - p2)^2 = 0.01
n per group = 249.7 -> 250
total = 500; with 20% dropout: 500 / 0.8 = 625Two points are worth noting. First, the answer is sensitive to the control rate as well as to the difference. If the control rate turns out to be 20% rather than 25%, and the treatment still cuts it by 40% in relative terms (to 12%), the absolute difference shrinks to 8 points and the requirement rises to about 330 per group. This is why blinded re-estimation of the pooled event rate is common in event-driven programmes. Second, this is the uncorrected normal approximation. The Fleiss continuity correction, n′ = (n/4)·[1 + √(1 + 4/(n·|p₁ − p₂|))]², raises the figure to about 270 per group. Protocols should say which version was used.
For survival endpoints analysed with the log-rank test (spec §4.3), power depends on the number of events, not on the number of subjects. For 1:1 allocation, Schoenfeld's approximation gives D = 4 · (Z1−α/2 + Z1−β)² / (ln HR)². For a hazard ratio of 0.75 at α = 0.05 and 80% power, that is 4 × 7.85 / 0.0828 ≈ 380 events. The number of subjects then follows from the expected event rate, accrual and follow-up. The WIA-BIO-010 v1.0 formulas do not cover this step, and the SDK has no time-to-event sizing function.
calculateSampleSize() takes the binary branch when controlRate and treatmentRate are given and no standardDeviation is. Its code squares the two terms separately and adds them, computing (A² + B²)/(p₁ − p₂)² instead of (A + B)²/(p₁ − p₂)². For the example above, that gives (1.2293 + 0.2223)/0.01 = 145.2 → 146 per group instead of 250, a 42% under-estimate. At 146 per group, the true power for this comparison is about 57%, not the 80% the design claims. The effectSize argument is ignored on this branch. Until the code is corrected, do not use the SDK's binary output for design. Section 2.8 lists this with the other deviations.
Solving the continuous-endpoint formula for 1 − β instead of n gives the power of a fixed design:
SE = sigma * sqrt(2 / n)
power = Phi( Delta / SE - Z[1-alpha/2] ) (+ a negligible opposite-tail term)Here Φ is the standard normal cumulative distribution function. Δ/SE is the non-centrality, the expected value of the test statistic when the true difference is Δ. Power is the chance that a normal variable centred there lands beyond the critical value. The formula is quick to evaluate, and it shows why doubling n raises power much less than doubling Δ: n enters only through its square root.
Spec §4.2 lists power values for n = 100 per group, σ = 1.5 and α = 0.05 of 42%, 80%, 96% and 100% for Δ = 0.3, 0.5, 0.7 and 1.0. These values do not follow from the formula. With these inputs SE = 1.5 × √0.02 = 0.2121, and:
| Δ | Δ / SE | Δ / SE − 1.96 | Power (correct) | Spec §4.2 value (replace) |
|---|---|---|---|---|
| 0.3 | 1.414 | −0.546 | 29% | 42% |
| 0.5 | 2.357 | 0.397 | 65% | 80% |
| 0.7 | 3.300 | 1.340 | 91% | 96% |
| 1.0 | 4.714 | 2.754 | 99.7% | 100% |
A quick consistency check shows the spec table cannot be right. The spec's own §4.1 example needs 142 subjects per group to reach 80% power for Δ = 0.5 and σ = 1.5, so 100 per group cannot also give 80%. The recomputed curve below replaces the spec's table in the rest of this book.
Figure 2-1. Power against the true difference Δ for σ = 1.5 and two-sided α = 0.05. The marked points are the recomputed §4.2 values for 100 subjects per group. The dashed curve shows that doubling to 200 per group raises power at Δ = 0.5 from 65% to about 92%. At Δ = 0 the curve starts at α = 5%, the probability of a false positive.
The following standalone Python script uses only the standard library (statistics.NormalDist, Python 3.8 or later). It reproduces every design number in sections 2.2–2.4 with exact normal quantiles rather than the rounded 1.96 and 0.84. Use it as an independent reference when you check a protocol or an implementation.
from math import ceil, sqrt
from statistics import NormalDist
Z = NormalDist()
def n_continuous(sigma, delta, alpha=0.05, power=0.80):
z_a = Z.inv_cdf(1 - alpha / 2)
z_b = Z.inv_cdf(power)
return (z_a + z_b) ** 2 * 2 * sigma ** 2 / delta ** 2
def n_binary(p1, p2, alpha=0.05, power=0.80):
z_a = Z.inv_cdf(1 - alpha / 2)
z_b = Z.inv_cdf(power)
p_bar = (p1 + p2) / 2
a = z_a * sqrt(2 * p_bar * (1 - p_bar))
b = z_b * sqrt(p1 * (1 - p1) + p2 * (1 - p2))
return (a + b) ** 2 / (p1 - p2) ** 2
def power_continuous(n, sigma, delta, alpha=0.05):
z_a = Z.inv_cdf(1 - alpha / 2)
return Z.cdf(delta / (sigma * sqrt(2 / n)) - z_a)
raw = n_continuous(sigma=1.2, delta=0.5)
per_group = ceil(raw)
print(f"A.1 raw n/group = {raw:.2f}")
print(f"A.1 per group = {per_group}, total = {2 * per_group}")
print(f"A.1 with 20% loss = {ceil(per_group / 0.8)} per group")
print(f"4.1 per group = {ceil(n_continuous(sigma=1.5, delta=0.5))}")
print(f"binary 25% vs 15% = {ceil(n_binary(0.25, 0.15))} per group")
for d in (0.3, 0.5, 0.7, 1.0):
print(f"power n=100 sigma=1.5 delta={d}: {power_continuous(100, 1.5, d):.3f}")Expected output:
A.1 raw n/group = 90.42
A.1 per group = 91, total = 182
A.1 with 20% loss = 114 per group
4.1 per group = 142
binary 25% vs 15% = 250 per group
power n=100 sigma=1.5 delta=0.3: 0.293
power n=100 sigma=1.5 delta=0.5: 0.654
power n=100 sigma=1.5 delta=0.7: 0.910
power n=100 sigma=1.5 delta=1.0: 0.997The raw figure 90.42 differs slightly from the hand calculation's 90.32 because exact quantiles (1.95996 and 0.84162) replace the rounded ones. Both round up to 91. The SDK's calculatePower() with a standardDeviation implements the same formula, using a polynomial approximation of Φ that is accurate to several decimal places, and returns the same four values. Without a standard deviation it falls back to 0.5 + 0.4 · (n/100) · Δ. That is a heuristic, not a power calculation: for n = 100 and Δ = 0.5 it returns 0.70 whatever the variability of the endpoint. The CLI's calc-power command uses the same heuristic, so its output should not be quoted as a result.
The primary analysis applies one of the tests of spec §4.3 to the locked data. The confidence interval of spec §4.4 then says how large the effect plausibly is. A p-value alone does not answer the clinical question, which is why regulators expect the estimate and its interval alongside it.
s_p^2 = ((n1 - 1) s1^2 + (n2 - 1) s2^2) / (n1 + n2 - 2)
t = (xbar1 - xbar2) / sqrt(s_p^2 * (1/n1 + 1/n2)), df = n1 + n2 - 2
95% CI: (xbar1 - xbar2) +/- t[0.975, df] * SEAs an illustration only, suppose the A.1 trial completes with 91 evaluable subjects per arm. The mean HbA1c change is −0.9 (SD 1.1) on treatment and −0.4 (SD 1.2) on control. The pooled SD is 1.151, SE = 1.151 × √(2/91) = 0.171, and t = −0.5 / 0.171 = −2.93 with 180 degrees of freedom, giving a two-sided p ≈ 0.004. With t0.975,180 = 1.973, the 95% CI for the difference is −0.5 ± 0.337 = (−0.84, −0.16). In practice the primary analysis of a change-from-baseline endpoint is usually an ANCOVA adjusted for baseline and stratification factors, which reduces the variance further. The unadjusted t-test is the textbook special case.
The SDK's performHypothesisTest() implements the pooled t statistic correctly but has two shortcuts that matter for small trials. Its interval always uses 1.96 rather than the t quantile, which makes it too narrow when df is small. For df ≤ 30 its p-value comes from the expression 1 − ½·(df/(df + t²))df/2, which is not the t distribution. At the exact 5% critical values (t = 2.228 with 10 df, t = 2.042 with 30 df) it returns two-sided p-values of about 0.13 and 0.14 instead of 0.05. A small, genuinely significant study would be reported as non-significant. Use it only for large samples, or replace the CDF with a proper implementation.
For a 2×2 table with events a and c out of n₁ and n₂, spec §4.3 gives the Pearson χ² = Σ(O − E)²/E with one degree of freedom, and spec §4.4 gives log-scale intervals for the risk ratio and odds ratio:
RR = (a/n1) / (c/n2) SE[ln RR] = sqrt(1/a - 1/n1 + 1/c - 1/n2)
OR = (a*d) / (b*c) SE[ln OR] = sqrt(1/a + 1/b + 1/c + 1/d)
95% CI = exp( ln(estimate) +/- 1.96 * SE )Appendix A.2 uses 30 events in 200 treated subjects and 50 events in 200 controls. The point estimate RR = 0.15 / 0.25 = 0.60 is correct. The interval is not. The terms under the root add to 0.0333 − 0.005 + 0.02 − 0.005 = 0.0433, whose square root is 0.208, not the 0.217 printed. Carrying the error through gives a margin and an interval that are both too wide:
SE[ln RR] = sqrt(0.04333) = 0.2082
ln(0.60) = -0.5108
margin = 1.96 * 0.2082 = 0.4080
ln-scale CI = (-0.9188, -0.1028)
95% CI for RR = (0.40, 0.90) [spec prints (0.39, 0.92)]The same table gives the other common summaries. The risk difference is −0.10, with Wald 95% CI (−0.178, −0.022). The odds ratio is (30 × 150)/(170 × 50) = 0.53, with 95% CI (0.32, 0.88). The Pearson χ² is 6.25 (expected events 40 per arm), p ≈ 0.012. All four measures agree on direction and significance, but they describe the effect differently. The odds ratio is always further from 1 than the risk ratio when events are not rare, so it should not be read as a risk ratio. The summary measure that counts is the one named in the estimand.
The log-rank statistic of spec §4.3, χ² = [Σ(O₁ − E₁)]² / ΣV₁ with one degree of freedom, compares the observed number of events in one arm with the number expected if the hazards were equal. At each event time the expectation and variance are computed from the risk sets. It is most powerful when hazards are proportional. When curves cross or effects are delayed, as with some immunotherapies, the protocol should pre-specify an alternative such as a weighted log-rank test or restricted mean survival time. The hazard ratio and its interval usually come from a Cox model fitted alongside.
Each additional hypothesis tested at α adds another chance of a false positive. With m independent tests, each at 0.05, the probability of at least one false positive is 1 − 0.95m: 23% for m = 5. Confirmatory trials therefore control the family-wise error rate (FWER) across every hypothesis that could support a claim, including multiple primary endpoints, several doses against one control, key secondary endpoints and subgroup claims. Spec §4.5 names three procedures.
| Procedure | Rule (p-values sorted p₍₁₎ ≤ … ≤ p₍ₘ₎) | Controls | Use in confirmatory claims |
|---|---|---|---|
| Bonferroni | Reject H₍ᵢ₎ if p₍ᵢ₎ ≤ α/m | FWER | Acceptable; simple but conservative |
| Holm step-down | Reject H₍₁₎ … while p₍ᵢ₎ ≤ α/(m − i + 1); stop at first failure | FWER | Acceptable; uniformly more powerful than Bonferroni |
| Benjamini–Hochberg | Find largest i with p₍ᵢ₎ ≤ (i/m)·q; reject H₍₁₎ … H₍ᵢ₎ | False discovery rate (FDR) | Not for confirmatory claims; suited to exploratory and biomarker screening |
Suppose, as an illustration, four hypotheses return p-values 0.004, 0.020, 0.030 and 0.041 at an overall α (or q) of 0.05.
| Rank i | p₍ᵢ₎ | Bonferroni 0.05/4 | Holm α/(m − i + 1) | BH (i/4)·0.05 |
|---|---|---|---|---|
| 1 | 0.004 | 0.0125 → reject | 0.0125 → reject | 0.0125 → reject |
| 2 | 0.020 | 0.0125 → no | 0.0167 → no, stop | 0.025 → reject |
| 3 | 0.030 | 0.0125 → no | (not tested) | 0.0375 → reject |
| 4 | 0.041 | 0.0125 → no | (not tested) | 0.05 → reject |
Bonferroni and Holm both support one claim. Benjamini–Hochberg supports four, but it only promises that on average no more than 5% of the rejected hypotheses are false. It does not promise that none of them is. That guarantee is not strong enough for a labelling claim, which is why regulatory guidance on multiple endpoints builds on FWER-controlling methods: Bonferroni and Holm, fixed-sequence (hierarchical) testing, gatekeeping and graphical procedures. Fixed-sequence testing is often the most efficient choice when endpoints have a natural order. Each hypothesis is tested at the full α only if every hypothesis before it was rejected, and the order must be fixed in the protocol. Choosing the order after seeing the data breaks the error control.
Long trials are usually examined by a data safety monitoring board (DSMB) at planned interim analyses. Chapter 6 covers the governance of those reviews. This section covers the statistics. Testing the accumulating data at the full α at every look would inflate the Type I error, so the protocol pre-specifies a group-sequential design that spreads α over the looks.
O'Brien–Fleming boundaries set very strict critical values early, when information is scarce, and relax them towards the end. The final analysis is then tested at almost the nominal level. For three equally spaced looks at overall two-sided α = 0.05, the standard boundaries (tabulated in Jennison and Turnbull, which spec §10.3 cites) are:
| Look | Information fraction | Critical |Z| | Nominal two-sided p to stop |
|---|---|---|---|
| 1 | 1/3 | 3.471 | ≈ 0.0005 |
| 2 | 2/3 | 2.454 | ≈ 0.014 |
| 3 (final) | 1 | 2.004 | ≈ 0.045 |
Spec §4.6 lists α₁ = 0.00001, α₂ = 0.014, α₃ = 0.045 and then "final α = 0.05". The first value does not match an O'Brien–Fleming boundary, and the last line suggests a fourth test at 0.05 that would push the overall error above 5%. The final analysis of this design is tested at a nominal level of about 0.045, and the three looks together spend exactly 0.05. In practice, most protocols use the Lan–DeMets α-spending function of O'Brien–Fleming type, α(t) = 2 − 2Φ(Z1−α/2/√t). It gives nearly the same boundaries but allows the number and timing of looks to change without losing error control. Either way, the design software's output, not a hand-typed list, should be stored with the protocol.
Futility stopping protects participants from continuing in a trial that cannot succeed. Conditional power (CP) is the probability of a significant final result given the interim Z statistic Zt at information fraction t and an assumed drift θ, where θ is the expected final Z:
CP(theta) = 1 - Phi( (Z[1-alpha/2] - Z_t * sqrt(t) - theta * (1 - t)) / sqrt(1 - t) )
"current trend": theta = Z_t / sqrt(t)
"design alternative": theta = Z[1-alpha/2] + Z[1-beta] (= 2.80 for 5%/80%)As an illustration, halfway through (t = 0.5) with Zt = 0.8, CP under the current trend is about 12%. Under the design alternative it is about 50%. Both numbers are correct. They answer different questions, and the protocol must say which one the futility rule uses. Spec §4.6 and §9.3 use a threshold of CP < 20%. That is a common convention, not a regulatory rule. Futility boundaries are normally non-binding: the DSMB may continue despite crossing them. Because stopping for futility can only lower the Type I error, non-binding rules leave α intact, whereas a binding rule would have to be accounted for in the design.
Spec §9.3 also mentions stopping when the Bayesian predictive probability of success falls below 10%. Instead of fixing θ, this approach averages conditional power over the posterior distribution of the effect given the interim data and a stated prior. It is therefore less sensitive to the choice between "current trend" and "design alternative". The threshold, the prior and the success criterion at the final analysis must all be pre-specified, and the operating characteristics (Type I error, power, expected sample size) must be shown by simulation. Chapter 6 describes how these rules are written into the DSMB charter and how stopping decisions are recorded.
Statistical code in a trial system has to be validated like any other computerized system. Chapter 8 builds a full test suite. The table below records what the v1.0 SDK returns today for the examples in this chapter, based on reading its source. Treat every mismatch as a defect to fix or to guard around.
| Case | Correct value | SDK behaviour | Status |
|---|---|---|---|
| A.1 continuous (σ 1.2, Δ 0.5, α 0.05, 80%) | 91 / 182; 228 with 20% dropout | 91 / 182; adjustedSampleSize 228 | Matches |
| §4.1 continuous (σ 1.5, Δ 0.5) | 142 / 284; 355 | 142 / 284; 355 | Matches (spec text wrong) |
| Same as A.1 with α = 0.01 | 135 per group | 91 — Z fixed at 1.96 for any alpha | Defect |
| A.1 with power 0.95 | 150 per group | 121 — any power ≥ 0.9 uses 1.28, any other uses 0.84 | Defect |
| Binary 25% vs 15% | 250 per group | 146 — terms squared separately | Defect |
| Power, n 100, σ 1.5, Δ 0.5 | 0.654 | ≈ 0.654 | Matches |
| Power without σ | undefined (needs σ) | 0.70 heuristic | Defect — should throw B005 |
| t-test p-value, df ≤ 30, t at 5% critical value | 0.05 | ≈ 0.13–0.14 | Defect |
Until these are fixed, wrap the SDK: reject alpha values other than 0.05 and power values other than 0.80 or 0.90, compute binary sample sizes and small-sample p-values with a validated library, and require a standard deviation for power. Record the reference values from the script in section 2.4 as regression tests.
@wia/bio-010 SDK and CLI, have been checked against an independent calculation.