Every study, searchable

Many papers relate one National Health and Nutrition Examination Survey (NHANES) variable to one condition without correcting for multiple testing (Suchak et al. (2025)). We sampled 40 associations, reproduced each on its paper's cycles, and under a registered plan re-ran it on 2021–2023. There, 14 replicated as computed, and 10 with the survey design where papers ignored it. Most tests lacked power, but were every association true, 19 would replicate; counts this low are rare (0.047, or 0.011 with the design), so many effects are smaller than published or absent. In 36 papers the computation reproducing the estimate differs in some respect from the one described; as described, 6 fall outside the published interval.

• Published.Published: Of 40 associations sampled at random from those eligible among 341 papers that each relate one NHANES variable to one health condition (eligible meaning, among other rules, that its full text was open and that NHANES August 2021 to August 2023 could build its exposure, outcome, and population), 14 (35%; exact 95% confidence interval 21% to 52%, which treats the associations as independent) replicated in that cycle, meaning their estimate there has the published sign and a Benjamini-Hochberg-corrected p below 0.05, with each analysis computed as its paper's was. Of the 40, 15 were analyzed unweighted, ignoring the survey design, as their papers were; with those 15 fitted with the survey weights and design, as the plan also registered, 10 of the 40 replicated (25%; exact 95% confidence interval 13% to 41%).
• Published.Published: At an unadjusted two-sided alpha of 0.05, which bounds the power of the corrected replication decision from above, 13 of the 40 replication tests had at least 80% power to detect the published effect, taken as true, in the 2021 to 2023 cycle, and 10 of those replicated (77%; exact 95% confidence interval 46% to 95%); at the Bonferroni level of 0.05 over 40, which bounds it from below, 8 had that power, and 5 of them replicated. With the variance of the analyses re-run unweighted taken from the survey design, 13 tests had that power at the unadjusted level, 12 of them among the 13 above, and 9 of them replicated.
• Published.Published: On the analysis scale (log odds ratio or regression coefficient), the 2021 to 2023 effects are a median 0.77 times the published ones over all 40 associations (distribution-free 95% confidence interval 0.32 to 1.05) and 0.91 times (0.32 to 1.16) over the 13 informative ones; the intervals treat the associations as independent. The median is 0.53 over the 15 analyses re-run unweighted and 0.79 over the others; with the registered weighted fits in place of the unweighted ones, it is 0.85 (0.35 to 1.08).

12 more claims

Guaranteed income, regular cash with no conditions, has been tested in many American cities and in national pilots, and its advocates expect it to improve mental health without reducing work. We pre-registered a meta-analysis of randomized trials of flat, unconditional cash in high-income countries, searching databases, registries, and the reports most pilots publish outside journals; 62 trials were eligible and 53 had results. About a year into payments, mental health was better in the cash arms by 0.09 standard deviations (CI 0.041 to 0.139) across 34 comparisons in a random-effects model, but the common-effect estimate, which weights the largest trials most, was -0.002 (-0.023 to 0.018), and smaller trials reported larger effects. Employment changed by 0.599 percentage points (-2.533 to 3.731), and the mental-health effect did not fade during payments. Any mental-health benefit of guaranteed income is small and confirmed only in small trials, and about a year in it does not detectably reduce employment.

• Published, Reproduced.Published, Reproduced: In 34 comparisons from 32 randomized trials of recurring unconditional cash in high-income countries (16,834 people analysed), mental health about 12 months into payments was better in the cash arms by a pooled standardized mean difference of 0.090 (95% CI 0.041 to 0.139; Holm-corrected p = 0.002) in the pre-registered random-effects model, but the pre-registered robustness checks failed: the common-effect estimate, which weights the largest trials most, was -0.002 (-0.023 to 0.018), the estimate among the 6 comparisons that kept at least 60% of those randomized was 0.043 (-0.077 to 0.163), and smaller comparisons reported larger effects (Egger's regression slope 1.54, 95% CI 0.75 to 2.33; p < 0.001).
• Published, Reproduced.Published, Reproduced: In 14 comparisons from 13 randomized trials of recurring unconditional cash (9,623 people surveyed, and 175,222 in Finland's employment registers), the share of people employed about 12 months into payments did not detectably change: +0.60 percentage points (95% CI -2.53 to 3.73; Holm-corrected p = 1.000), an interval that rules out a reduction larger than 2.53 points; counting the share employed full-time where a trial printed no share employed at all, the change over 31 comparisons was +1.13 points (-1.14 to 3.40).
• Published, Reproduced.Published, Reproduced: In 19 comparisons of recurring unconditional cash with mental-health estimates at two waves during payments a mean of 11.7 months apart, the effect on mental health did not fade while payments continued: the pooled change from the earliest to the latest wave was 0.000 standard deviations (95% CI -0.049 to 0.048; Holm-corrected p = 1.000), the effects at the two waves being 0.048 and 0.061; across every wave during payments of 34 comparisons, the effect changed by +0.015 standard deviations per year (-0.034 to 0.064).

4 more claims