# Pre-registered plan: serious adverse events and deaths in randomized placebo-controlled trials of preventive vaccines

## Question

Each vaccine's own trials report how many participants had a serious adverse event (one that results in death, is life-threatening, requires or prolongs hospitalization, or causes lasting disability) in the vaccine and placebo arms, and each finds the two rates similar. Nobody has pooled those tables across vaccines. Reviews pool within one vaccine or one disease; the widest, of vaccines with novel adjuvants, pooled 26 trials in older adults (Baay et al., Vaccine 2018;36:4207) and 29 in children (Stassijns et al., Vaccine 2016;34:714). A crowd-sourced list of vaccine trials with placebo controls (Scott and Spellberg, 2025) names the trials without their counts.

Several questions are argued over, in public and in the literature, that a pooled randomized estimate bears on:

- whether vaccines as a class cause serious harm that single trials are too small to show, and whether they were tested against inert placebos at all;
- whether aluminium adjuvants (Krauss et al., BMJ Open 2022;12:e058795: serious adverse events risk ratio 1.18, 0.97 to 1.43, very low certainty) or the mRNA platform (Fraiman et al., Vaccine 2022;40:5798: an excess of serious adverse events of special interest in two trials) carry such harm;
- whether live vaccines lower, and non-live vaccines raise, illness and death from causes other than their target disease, the "non-specific effects" hypothesis (Higgins et al., BMJ 2016;355:i5170; Benn et al., iScience 2023;26:106733);
- whether vaccines given to older adults lower cardiovascular events, as observational studies report for influenza, zoster and other vaccines, with reductions of a tenth to a quarter, and as small randomized trials of influenza vaccine in people with heart disease found (Behrouzi et al., JAMA Netw Open 2022;5:e228873).

Every placebo-controlled trial that posted results to ClinicalTrials.gov lists, for each arm, the participants with any serious adverse event, every serious adverse event by term, and (in records since 2017) deaths from any cause. Posting is required by law for most of them, whether or not they were published, and randomization, blinding and placebo make the arms comparable in everything but the vaccine.

**Questions.**

1. Across every such trial of a preventive vaccine, do vaccine recipients have more or fewer serious adverse events of any kind (P1), deaths from any cause (P2), or serious events other than infections (P4) than placebo recipients, and how much difference do the trials exclude?
2. In trials that enrolled only adults aged 50 and over, do vaccines change major cardiovascular events (P3)?
3. Do the answers to question 1 differ by what the placebo held, by the vaccine's platform and adjuvant, by age, or in pregnancy?
4. For outcomes named in advance (family H), in what the trials allow one to detect, and without hypotheses for each organ system, does anything differ between vaccine and placebo in one half of the trials and again in the other?
5. Do live and non-live vaccines differ in serious events from causes other than their target?

## Data

ClinicalTrials.gov's API (v2), retrieved on 2026-11-09: interventional studies with posted results (`aggFilters=results:with,studyType:int`) that name a vaccine (any of "vaccine", "vaccines", "vaccination", "vaccinated", "immunization", "immunisation" in the brief or official title, brief summary, keywords, intervention names, other names or descriptions, or arm labels or descriptions, or the intervention MeSH term "Vaccines") and either are listed as randomized and name a placebo (the words "placebo" or "saline" anywhere in the record, or an arm of type placebo comparator or sham comparator), or are not listed as randomized but have an arm of type placebo comparator or sham comparator, since a few randomized trials are registered with another allocation. The two queries are in `data/search.json` with the 874 studies they found. Each study's full record is in `data/ctgov/<NCT>.json`, byte for byte as the API returned it, and `ctgov_digests.json` holds each record's SHA-256. ClinicalTrials.gov data are public domain.

Trials whose results aren't posted are not in this study, however large: among them the Shingles Prevention Study, the pivotal trials of Rotarix in Latin America, of RSVpreF in older adults, and of NVX-CoV2373.

## Eligible trials

1. **By the record's structured fields:** at least two adverse-event groups. This leaves 869 of the 874.
2. **By two coders, under `screening_codebook.md`:** whether the trial has adverse-event groups that compare a preventive vaccine against an infectious disease with placebo with nothing else different; which groups those are (doses, formulations, lots, schedules and cohorts pooled; one comparison per background; infants of women vaccinated in pregnancy as their own comparison); what the placebo held (inert as stated, an adjuvant or other active part, or not stated); each vaccine group's platform and adjuvant; the target diseases; and whether the vaccine was given against a disease other than its own, to pregnant women, or to people enrolled for a medical condition. Therapeutic vaccines, vaccines against non-infectious targets, antibodies, active-control vaccines used as the comparator, no-treatment controls, and groups that aren't randomized arms over the placebo-controlled period are left out.

The coders were Claude Opus 5.5 (coder 1) and Claude Sonnet 5.5 (coder 2), each working alone from the codebook and, for each trial, its titles, conditions, phases, design, masking, age limits, arms, interventions, and adverse-event time frame and group titles and descriptions, with no counts, in batches of up to 60 trials. A third reader (Claude Fable 5.2, the model that designed this study), given the codebook, the record, both codings, and notes on the points the coders had reported the codebook left open (`coding/third_reader_notes.md`, written before any of these trials was decided), decided every trial on which the coders differed in anything the analysis reads, and gave its reason. Both codings, the third reader's decisions, and the agreement are in `coding/`; the decided screening is `data/screening.json`.

The coders agreed on eligibility for 848 of the 869 trials (κ = 0.95), chose identical groups for 477 of the 504 both found eligible, and agreed on everything the analysis reads for 751. The third reader decided the other 118; it had also been given two trials on which the coders differed only in the order in which they listed two platforms, and there the coders' coding stands. After its decisions, 514 trials are eligible. Of their comparisons, 501 from 491 trials, with 1,133,392 participants in their adverse-event tables, are of randomized participants against a placebo not stated to hold an adjuvant or other active part (the **main set**): 315 against a placebo stated to be inert (561,002 participants) and 186 against a placebo of unstated composition (572,390). A further 23 comparisons (34,904 participants) are against a placebo holding an adjuvant or other active part, and 15 (21,094) are of infants born to women vaccinated in pregnancy; both are analysed apart.

**From the structured fields**, for each trial: its age band, from the eligibility limits (infants: an upper limit of 2 years or less; children: an upper limit under 18 years; older adults: a lower limit of 50 years or more; adults: a lower limit of 18 years or more; mixed: anything else); whether masking is double or more; whether the phase is 3 or 4; and whether the lead sponsor is industry. Infants born to vaccinated women are in the infants band.

## Outcomes and counting

**Participants.** For a comparison and an arm (its adverse-event groups pooled), participants with any serious adverse event are the groups' `seriousNumAffected` over `seriousNumAtRisk`, and deaths from any cause are `deathsNumAffected` over `deathsNumAtRisk`, where every group in the comparison reports them (records from before 2017 often give no deaths table). Both count each participant once. In the main set, 501 comparisons (1,133,392 participants) report any serious event and 366 (805,500) report deaths.

**Reports.** Every other outcome is a set of rows of the serious adverse event table (a term under an organ system), and its count is the sum over those rows of participants affected, capped at the participants at risk. A participant with two of an outcome's rows counts twice, so these are counts of reports, and their ratios compare reports per participant between arms. The variance below allows for this.

**Terms to tags.** Every distinct serious adverse event term in the 869 trials (6,393 after collapsing spaces and case) was given the outcome tags it meets, out of 42, by the same two coders working alone under `outcome_codebook.md`, each with only the terms and the organ systems they were reported under, in batches of up to 800. Terms on which they differed were decided by the third reader, with notes written beforehand (`coding/third_reader_notes_terms.md`). The coders gave identical tags to 6,305 of the 6,393 terms (κ = 0.939 over term-tag pairs), and the third reader decided 88. The decided coding is `data/term_coding.json`.

**Outcome definitions** are in `data/outcomes.json`: each outcome's rows (by tags, by organ system, by everything outside an organ system, or infections other than the vaccine's target, which leaves out of the infections organ system the rows tagged for the trial's target diseases as `target_tags` lists them) and the comparisons it is estimated in. Non-serious adverse events are not used, except injection-site pain as a check of the data, since trials report them only above a frequency threshold.

## Estimates

- **Main estimate:** the Mantel-Haenszel risk ratio of the vaccine arm against placebo, stratified by comparison, with a 95% confidence interval and a two-sided Wald p. Its variance is the wider of two: Greenland and Robins's, which treats each count as binomial, and a variance clustered by trial (a sandwich over each trial's contribution to the ratio's numerator and denominator, times G/(G − 1) for G trials with events), which leaves how a trial's count arises free and so holds for reports and for effects that differ between trials. Comparisons with no events in either arm add nothing to the ratio.
- **Risk difference:** the Mantel-Haenszel risk difference per 10,000 participants, with the wider of its Greenland-Robins and trial-clustered variances. Its upper bound is the largest excess the trials leave room for.
- **Sensitivity analyses** for the primary and named outcomes: random effects (DerSimonian-Laird on each comparison's log risk ratio, 0.5 added to every cell of a comparison with a zero cell), Peto's odds ratio, each half of the trials, every single trial left out, and, for outcomes counted in reports, the outcome's most frequent term alone.
- **Halves.** Trials are paired by size (participants in their comparisons, largest first, ties by NCT number), and one of each pair is assigned to half 1 by `numpy.random.default_rng(20261109)`; an unpaired last trial goes to half 1.
- An outcome with fewer than 10 participants or reports in all is reported as having too few events and is not tested.

## The four primary questions

All in the main set, each with a margin set here:

| | Outcome | Comparisons | Margin |
|---|---|---|---|
| P1 | Any serious adverse event (participants) | all | 0.95 to 1.05 |
| P2 | Death from any cause (participants) | all that report deaths | 0.90 to 1.10 |
| P3 | Major cardiovascular events: myocardial infarction, stroke, heart failure, cardiac arrest (reports) | trials of adults aged 50 and over | 0.90 to 1.10 |
| P4 | Serious events other than infections: every row outside the infections organ system (reports) | all | 0.95 to 1.05 |

P4 is there because a vaccine that works lowers serious infections, which could hide harm of another kind inside P1.

Holm's correction is applied over the four p-values. Each question gets one reading:

- **higher** or **lower** on vaccine: Holm p < 0.05;
- **no difference within the margin**: Holm p ≥ 0.05 and the 95% interval lies inside the margin;
- **inconclusive**: otherwise.

A reading of higher or lower is **robust** if the random-effects estimate has p < 0.05 on the same side, every leave-one-trial-out p is below 0.05, and both halves' estimates lie on that side. A reading of no difference within the margin is robust if the random-effects interval and every leave-one-trial-out interval also lie inside the margin.

From pooled counts blind to arm (`power.json`), a two-sided 5% test has 80% power to detect risk ratios beyond 0.98 and 1.02 for P1 (55,656 participants affected), 0.93 and 1.07 for P2 (6,602 deaths), 0.91 and 1.10 for P3 (3,651 reports), and 0.98 and 1.02 for P4 (59,730 reports), before allowing for the clustered variance.

## Kinds of trial and vaccine

Any serious adverse event and death are estimated, as risk ratio and risk difference, in each of the following, listed in `data/outcomes.json` (`strata`): placebo stated inert, placebo of unstated composition, and placebo holding adjuvant; each platform (live attenuated, viral vector, mRNA, DNA, inactivated, protein, polysaccharide or conjugate), taking the vaccine groups of that platform alone against the comparison's placebo; aluminium-adjuvanted vaccines, vaccines with another adjuvant, and vaccines with none, and the first two and mRNA again against placebo stated inert; infants, children, adults, older adults, and mixed ages, and children against placebo stated inert; pregnant women, and separately the infants born to them; people enrolled for a medical condition, and everyone else; double-blind or more; phase 3 and 4; industry and other lead sponsors; comparisons of 1,000 participants or more; COVID-19 vaccines and all others; vaccines given against a disease other than their own; and each target disease with at least 5 trials. These are descriptions with intervals, not tests: no p-value correction is applied and no reading is attached, except that the paper will say which intervals lie inside P1's and P2's margins.

## Named outcomes (family H) and checks of the data

`data/outcomes.json` lists 31 named outcomes (`named`), each with the comparisons it is estimated in: major cardiovascular events in all adult trials; myocardial infarction, stroke, heart failure, atrial fibrillation, pneumonia, and dementia in trials of adults 50 and over; myocarditis or pericarditis, overall and for mRNA vaccines; venous thromboembolism; Guillain-Barré syndrome; facial palsy; demyelinating disease; seizures, overall and in children; anaphylaxis; thrombocytopenia; immune-mediated disease, overall and for vaccines with a non-aluminium adjuvant; Kawasaki disease, asthma or wheezing in children; narcolepsy; appendicitis; intussusception and sudden infant death in infants; pneumonia; sepsis; preterm birth in pregnant women and among infants of vaccinated women; fetal death or miscarriage; and infections other than the vaccine's target. Each is estimated as above, with Benjamini-Hochberg q over those tested. The paper reports every one, with what a test could have detected (`power.json`).

**Positive controls**, effects the trials' own reports establish, which the pooled tables should show if arms and terms are assigned rightly: fewer serious COVID-19 events in COVID-19 vaccine trials (PC1), fewer serious dengue events in dengue vaccine trials (PC2), fewer serious gastroenteritis events in rotavirus vaccine trials (PC3), and more injection-site pain among non-serious events against inert placebo (PC4). If any is not in its direction with p < 0.05, the paper says so and treats every other estimate as suspect.

**Negative controls**, outcomes a vaccine should not change, which would show a difference between arms in follow-up or reporting: fractures in trials of adults 50 and over (NC1), the injury, poisoning and procedural complications organ system (NC2), and the neoplasms organ system (NC3). If any differs from 1 with p < 0.05, the paper says so.

## Organ systems

Every organ system with at least 10 reports in the main set is estimated in half 1, with Benjamini-Hochberg q over the organ systems, and again in half 2. An organ system **replicates** if q < 0.05 in half 1 and p < 0.05 in half 2, in the same direction. The estimate over all trials, with q over the organ systems, is reported beside them.

## Live against non-live vaccines

In the main set, taking the vaccine groups that are live attenuated alone, and those of non-live platforms alone (mRNA, DNA, inactivated, protein, polysaccharide; viral vectors are in neither), each against its comparison's placebo: any serious adverse event, death, the infections organ system, and infections other than the vaccine's target, in all ages and in trials of children, each with the ratio of the live to the non-live risk ratio and its z test. The non-specific effects hypothesis predicts a ratio below 1. These trials are mostly in high-income countries and adults, so they cannot test it where it was proposed, in children in places where many die.

## Registry counts against published counts

After the registered analysis, for the 15 comparisons of the main set with the most participants, a reader with web access who is not shown the registry's counts will look for the trial's main publication, regulator's review, or sponsor's report, and take from it the participants with at least one serious adverse event and the deaths in the vaccine and placebo arms over the placebo-controlled period. The paper will list both sets of counts and repeat P1 and P2 with the published counts in place of the registry's where they were found. This is a description of how far the registry's tables agree with other accounts of the same trials; its code reads a file that doesn't exist yet and will be written after registration.

## What was seen before this plan was registered

- The record format, and pooled counts over both arms and all groups of the 869 trials: participants with any serious adverse event, deaths, and participants affected per term and per organ system, to see whether the study was feasible.
- The coders' reports of what the codebooks left unclear, which the third reader's notes answer. No count was shown to any coder or to the third reader.
- The analysis code was written and tested with each comparison's arms swapped at random (`code/analyze.py --blind`), first on one coder's screening and the other's term coding, then on the decided codings. Its estimates under that swap say nothing about vaccines and were used only to check that the code runs; the same run gave `power.json`, whose numbers don't depend on which arm is which. No count by arm, and no estimate with arms as they are, was computed or looked at.
- The published results of the large trials, as anyone in the field knows them: each reported similar rates of serious adverse events in its two arms.
- The fourth primary question (P4) and the lower respiratory infection tag were added while the coders were working, before any coding was compared and before the blinded run.
- The search at first took only trials the registry lists as randomized, and the structured rule also dropped designs it lists as single group. After the first blinded run on the decided codings, a look at how the largest trials had been screened (by enrollment and participants at risk, with no count of events) showed that the second rule had dropped randomized placebo-controlled trials whose records give that design, the Rotavirus Efficacy and Safety Trial among them. Both restrictions were removed, and the 20 trials this added were screened, and their 77 new terms coded, by the same coders and third reader under the same codebooks and notes, before the blinded run that gave `power.json`.
- The screeners of one coder shared a scratch folder and reported overwriting one another's display scripts (scripts that print trials, not codings); each then worked from files of its own.

## Files

- `plan.md`, `screening_codebook.md`, `outcome_codebook.md`: this plan and the codebooks.
- `coding/`: both screenings, both term codings, the third reader's notes and decisions with reasons, the terms as coded (`terms_input.json`), and the agreement.
- `data/`: what the analysis reads, as registered: `screening.json`, `term_coding.json`, `outcomes.json`, `parameters.json`, and `structured.json` (what the structured fields kept).
- `ctgov_digests.json`: the SHA-256 of each of the 874 records, so the data analysed are those registered.
- `code/`: the analysis (`analyze.py`) and its command (`run`), with `requirements.txt` (Python 3.12, numpy 2.5.3, scipy 1.18.1).
- `power.json`: what the blinded run gave that doesn't depend on which arm is which.

## Deviations

Any departure from this plan will be listed in the bundle's `deviations.json` with its reason.
