Methods
The verification bundle
Every power calculation the tool produces can be exported as a complete computational audit: every matrix in the derivation, plus an R script that reproduces the standard error independently. Nobody has to trust the software.
Why this exists
Sample size calculations are among the least reproducible parts of a trial protocol. A protocol typically states a number, an assumed effect size, an ICC, and a software name. A reviewer cannot check the arithmetic, cannot see the assumed covariance structure, and cannot tell whether the reported figure came from the design as finally specified or an earlier draft.
Trials units are right to be cautious about a browser-based calculator for exactly this reason. The verification bundle is the answer: it makes the computation fully inspectable, so the tool’s trustworthiness stops being something you have to take on faith.
What the bundle contains
| Object | Description |
|---|---|
X | Design matrix, one row per cluster-period, columns for intercept, treatment, and period effects |
Sigma | Model-implied covariance matrix under the specified correlation and sampling structure |
M | Information matrix \(X^{\top}\Sigma^{-1}X\) |
Minv | Its inverse, whose treatment diagonal element is the model-based variance |
bread, meat | Components of the sandwich variance, when a robust estimator is selected |
V_working | Working covariance used for estimation, where it differs from the truth |
Sigma_true | Assumed true covariance, used in the meat of the sandwich |
beta, theta | Fixed effect and covariance parameter vectors |
se, dof, power | Resulting standard error, degrees of freedom, and power |
| Solver diagnostics | Target versus achieved ICC, IAC and baseline; raw variance components; iteration count and convergence status |
| Data matrix | The cluster-period level dataset the model was built from |
| R script | Reads the exported matrices, recomputes the standard error, and compares it against the reported value |
The solver diagnostics
For binary and count outcomes, the reported ICC and IAC are targets on the observed scale that must be translated into random effect variances on the link scale. The bundle reports both the targets and what was actually achieved, along with the raw variance components and whether the solver converged.
This matters because a small discrepancy between target and achieved correlation is normal and acceptable, while a large one means the requested combination was near the edge of what the model can represent. Exposing both lets a reviewer judge for themselves rather than relying on a warning threshold chosen by the software. Background on why this arises →
Using it in a submission
- In a protocol appendix. Include the R script and the reported standard error. A statistical reviewer can rerun it in under a minute.
- In a CTU quality process. The bundle documents every assumption explicitly, including ones a protocol paragraph would omit — the exact covariance structure, the estimator, the degrees of freedom method.
- For your own records. When someone asks in eighteen months why the sample size is what it is, the bundle answers the question precisely, including which version of the design it referred to.
The export button is disabled while results are marked stale, so a bundle always corresponds to the design and parameters currently on screen rather than to an earlier state. If the button is greyed out, press Recalculate first.