Study
Opening the study…
Study
Opening the study…
Where the work did something other than what it follows, its registered plan or the claim it replicates, as its author says.
From its pre-registered plan, under plan/
Done differently
Two corrections to how code/analyze.R chooses one estimate per comparison, outcome, and wave, made after registration and before the results were seen (a third reader of the extraction found them): ties were broken by each entry's label sorted as text, so a trial's tenth entry came before its ninth, and are now broken by the order the entries were listed; and the plan's rule that an estimate adjusted for covariates comes before an unadjusted one was applied only between the kinds 'adjusted difference' and 'difference', and now also applies within a kind, using each entry's 'adjusted' field (it matters for estimates printed in standard-deviation units). The registered code is in plan/code/analyze.R; the code that ran is code/analyze.R, which differs only in these lines.
Bears on C1, C2, C3, C4, C5, and C6
Not stated
Two trials' reports print, for some outcomes, only the effect of claiming or receiving the payment, estimated by instrumental variables, rather than the difference between the arms as randomized that the extraction codebook asks for: the COVID-19 Cash Transfer Study II (seven estimates; its mental-health composite, which it also prints as an intention-to-treat difference, is kept in that form) and the Chicago emergency rental assistance lotteries (four estimates). These 11 estimates were left out, listed by trial, outcome, and value in work/assemble_extraction.py; the Chicago lotteries contribute no estimate.
Bears on C5
Not stated
OpenResearch, the largest trial, prints its mental-health effects only pooled over its three years of payments, and by year only in figures without numbers, so the registered rule (a wave near 12 months) leaves it out of P1 and P3. An exploratory analysis added after registration (code/summarize.R, results/S.json) standardizes its pooled Kessler 6 difference by its control group's standard deviation and adds it to P1's comparisons; it is reported beside P1, not in place of it.
Bears on C1
Done differently
Exploratory analyses added after the results were seen, in code/summarize.R: P1 pooled separately for estimates from scale scores and from shares above a cut-off, with the meta-regression coefficient of being a share; P1 by place (the United States against Europe); P1 among comparisons with at least 500 people analysed and the rest (the plan's threshold of 1,000 left four comparisons); and the number of people analysed in P1 and P2 with each shared control group counted once and Finland's register population apart. None is a planned test.
Bears on C1, C2, and C4