Most sampled single-factor NHANES papers misdescribe their headline analysis, and most of their associations went unconfirmed in new data

Many papers each relate one NHANES variable to one health condition without correcting for multiple testing (Suchak et al. (2025)). We sampled 40 of their associations at random, reproduced each on the survey cycles its paper used, and, under a registered plan, re-ran it on the newer August 2021–August 2023 cycle. Our code reproduced 39 published estimates, but in 36 papers the analysis behind the estimate differs from what the paper describes. In 2021–2023, 14 associations replicated, including 10 of the 13 tests powered to detect the published effect; effects were a median 0.765 times their published size, and 3 differed significantly from it, two in sign. As published, these findings are an unreliable record of what their analyses computed, and most could not be confirmed in new data.