Similar studies

Do the reported undercoverage rates for a binomial Wald confidence interval survive an independent computation without floating-point membership or probability comparisons? A direct integer binomial expansion and a squared interval-membership test reproduce the earlier claim on all 95904 grid pairs. Coverage falls below the lower threshold at 44412 pairs and below the nominal threshold at 88522 pairs. The respective fractions, rounded as in the earlier work, are 0.463088 and 0.923027. This is a numerical audit of known behavior, not a new interval method.

• Published, Reproduced.Published, Reproduced: An independent integer-arithmetic audit using the exact decimal critical value z = 1.959963984540054 finds Wald interval coverage below 0.93 at 44412 and below 0.95 at 88522 of the 95904 pairs n = 5,...,100 and p = 0.001,...,0.999, reproducing the earlier fractions 0.463088 and 0.923027 when rounded to six decimals.

How much false-positive error can arise when a nominally exact test treats clustered subsamples as independent? An exact finite benchmark enumerates null distributions for 75 balanced designs. In the selected beta-binomial design, the cell-level Fisher test rejects with probability 0.21567065919813758, compared with 0.0436374106894531 for a test using the known clustered null. With a single observation per donor, the Fisher rejection probability is 0.021484375. These are exact probabilities conditional on the specified toy model. They demonstrate calibration failure from ignoring dependence, without evaluating real biological pipelines or ranking practical alternatives.

• Published, Reproduced, Reviewed.Published, Reproduced, Reviewed: For the specified null of five independent donors per arm and twenty conditionally independent Bernoulli observations per donor with latent Beta(5,5) success probabilities (within-donor correlation 1/11), the cell-level two-sided probability-ordering Fisher test at nominal alpha=0.05 rejects with exact-model probability 0.215670659198, versus 0.043637410689 for the known-clustered-null absolute-difference-tail oracle. The finite 75-design grid has 31 cell-level Fisher probabilities above nominal, while all 75 oracle probabilities and all 15 independent-observation controls are at or below nominal. These are conditional toy-model calibration results, not evaluations of real biological analysis methods.

Many papers relate one National Health and Nutrition Examination Survey (NHANES) variable to one condition without correcting for multiple testing (Suchak et al. (2025)). We sampled 40 associations, reproduced each on its paper's cycles, and under a registered plan re-ran it on 2021–2023. There, 14 replicated as computed, and 10 with the survey design where papers ignored it. Most tests lacked power, but were every association true, 19 would replicate; counts this low are rare (0.047, or 0.011 with the design), so many effects are smaller than published or absent. In 36 papers the computation reproducing the estimate differs in some respect from the one described; as described, 6 fall outside the published interval.

• Published.Published: Of 40 associations sampled at random from those eligible among 341 papers that each relate one NHANES variable to one health condition (eligible meaning, among other rules, that its full text was open and that NHANES August 2021 to August 2023 could build its exposure, outcome, and population), 14 (35%; exact 95% confidence interval 21% to 52%, which treats the associations as independent) replicated in that cycle, meaning their estimate there has the published sign and a Benjamini-Hochberg-corrected p below 0.05, with each analysis computed as its paper's was. Of the 40, 15 were analyzed unweighted, ignoring the survey design, as their papers were; with those 15 fitted with the survey weights and design, as the plan also registered, 10 of the 40 replicated (25%; exact 95% confidence interval 13% to 41%).