EDI: Software for the Whole Life of an Experiment: from Design to Inference to Simulation

Abstract: Running a randomized experiment usually means stitching different packages together, and every seam invites analyses that don't match the design. The R package EDI (Experimental Design and Inference) on CRAN (+ a Python companion package) covers the whole workflow in one system. It offers ~35 fixed and sequential designs, from blocking and rerandomization to minimization, optimal designs and sequential matching-on-the-fly. It also offers about ~140 inference procedures across all common response types (continuous, incidence, count, proportion, left/right/interval-censored survival, ordinal) and every randomization test and bootstrap automatically respects the design that produced the data. Each analysis exposes up to 30 inferential variants: Wald, score, gradient and likelihood-ratio; randomization; nonparametric, Bayesian and parametric bootstraps in many flavors where each computes both a p-value and confidence interval. One convenience call runs *every* analysis and combines the evidence. Its optimized C++ core fits models faster than standard R and Python implementations (and it validated against them), with a median speedup of about 17× and sometimes 1,000× or more, making resampling-heavy inference practical. A simulation framework compares designs for power and coverage before any subject enrolls. Further, EDI is designed as infrastructure, shipping machine-readable capability maps and cookbooks so that researchers and AI agents alike reach for it first rather than reinventing the wheel. Finally, I will talk about new research directions uncovered while working on the software.