# Pre-registered plan: what GLP-1 receptor agonists do to serious adverse events in randomized placebo-controlled trials

## Question

GLP-1 receptor agonists (semaglutide, tirzepatide, liraglutide, dulaglutide and others) are now taken by a large share of adults in some countries. Their effects on glucose, weight, and cardiovascular and kidney outcomes are established by large randomized trials. Beyond those, most of what is said about them comes from observational studies. The broadest is the VA atlas of Xie, Choi and Al-Aly (Nature Medicine 2025;31:951–962, doi:10.1038/s41591-024-03412-w): in 215,970 veterans with diabetes who started a GLP-1 receptor agonist, compared with 1,203,097 who continued usual care, it reported lower risks of substance use disorders, psychotic disorders, seizures, dementia, thromboembolism, infections, pneumonia, respiratory failure and more, and higher risks of gastrointestinal disorders, hypotension, syncope, arthritis, kidney stones, interstitial nephritis and pancreatitis. Its authors and commentators say these associations need testing in randomized trials, which no observational design can replace.

Those randomized trials largely exist already. Every placebo-controlled trial of these drugs that posted results to ClinicalTrials.gov lists every serious adverse event (death, hospitalization, or the like) in each arm, by MedDRA term, whether or not anyone published it. Placebo-controlled trials randomize, blind, and ascertain events the same way in both arms. Their serious adverse event tables are complete (the frequency threshold applies only to non-serious events), and posting results is required by law for most of them, so the set is less selected by outcome than the journal literature.

**Questions.**

1. For each association the VA atlas named in its results (family A, 49 outcomes, cardiometabolic outcomes left out), what do randomized placebo-controlled trials show: the same effect, the opposite, a different size, or too little to say?
2. What do the trials show for other questions asked of these drugs (family B: gallbladder disease, intestinal obstruction, ischaemic optic neuropathy, diabetic retinopathy complications, thyroid, pancreatic, obesity-associated and all cancers, depression, fractures, falls, death from any cause, and any serious adverse event)?
3. Without hypotheses, which organ systems (family C) and which serious adverse events by preferred term (family D) differ between the drugs and placebo in one half of the trials, and replicate in the other half?

## Data

ClinicalTrials.gov's API (v2), retrieved on 2026-10-12: the search `query.intr=<term>` with `aggFilters=results:with,studyType:int` for 22 terms, the GLP-1 receptor agonists by name plus cagrilintide (to find its trials in combination with semaglutide), which found 539 interventional studies with posted results (`data/search.json`). Each study's full record is in `data/ctgov/<NCT>.json`, byte for byte as the API returned it. ClinicalTrials.gov data are public domain.

## Eligible trials

1. **By the record's structured fields:** randomized allocation, a parallel or factorial design, and at least two adverse-event groups. This leaves 432 of the 539 (56 not randomized, 40 crossover, 6 sequential, 4 single group, 1 with one adverse-event group).
2. **By two coders, under `screening_codebook.md`:** whether the trial has adverse-event groups that compare a GLP-1 receptor agonist with placebo with nothing else different, which groups those are (pooling doses and, in factorial or multi-background trials, one comparison per background), the molecules, and the population randomized (type 2 diabetes, type 1 diabetes, overweight or obesity without diabetes, other, mixed). Arms that add another active drug, active comparators, open-label no-treatment controls, and groups that aren't randomized arms over the randomized period are left out.

The coders were separate instances of Claude Opus 5.5 (the model that wrote this study), each working alone from the codebook and, for each trial, its titles, conditions, phases, masking, arms, interventions, and adverse-event group titles and descriptions, with no counts. Each coded the 432 trials in four batches. A third instance, given the codebook, the record, and both codings, decided every trial on which they differed in eligibility, comparisons, molecules, or population (14 of 432), and gave its reason. When both coders excluded a trial for different reasons, the first coder's reason is listed; the reason affects nothing. Both codings, the third reader's decisions, and the agreement are in `coding/`; the decided screening is `data/screening.json`.

The coders agreed on eligibility for 426 of the 432 trials (κ = 0.97), and on identical comparisons for 227 of the 232 both found eligible. After the third reader's decisions, 234 trials are eligible, with 257 comparisons and 161,309 randomized participants in them. The molecules' classes are in `data/molecules.json`, which adds one the coders named (CJC-1134-PC, an exendin-4 conjugate).

## Outcomes and counting

**Terms to outcomes.** Every distinct serious adverse event term in the 432 trials (4,644 after collapsing spaces and case) was coded to the outcomes of families A and B it meets, and to its MedDRA preferred term (`canonical`, which merges spellings), by two coders working alone under `outcome_codebook.md`, each with only the terms and the organ systems they were reported under, in four batches. Terms on which they differed (in outcomes or canonical term) were decided by a third reader as above, with its reason. The decided coding is `data/term_coding.json`, and both codings and the decisions are in `coding/`. The coders gave identical outcome sets to 4,569 of the 4,644 terms (κ = 0.976 over the 278,640 term–outcome pairs), and the third reader decided 163 terms: 75 on outcomes and 88 on the preferred term alone.

**Counts.** For a comparison and an arm (its adverse-event groups pooled), the count for an outcome is the sum, over the outcome's terms, of participants affected (`numAffected`), capped at the participants at risk. A participant with two of an outcome's terms is counted twice, so this is an upper bound on participants with the outcome. Two analyses below allow for that: the conservative variance, and the outcome's dominant preferred term alone, a count of participants with no double counting. Participants at risk are the group's `seriousNumAtRisk`. Death from any cause comes from the groups' `deathsNumAffected` where every group in the comparison reports it (older records often don't), and any serious adverse event from `seriousNumAffected`. A term's own denominator, which some trials give for sex-specific events, is not used. Organ systems (family C) are the reported `organSystem`, spaces collapsed and lower-cased. Preferred terms (family D) are the coded `canonical` terms. Non-serious ("other") adverse events are not used, since trials report them only above a frequency threshold.

## Estimates

- **Primary:** the Mantel–Haenszel risk ratio of the GLP-1 arm against placebo, stratified by comparison, with the Greenland–Robins variance, a 95% confidence interval, and a two-sided Wald p. Comparisons with no events in either arm add nothing. An outcome with fewer than 10 participants affected in all is reported as having too few events and is not tested.
- **Multiplicity:** Benjamini–Hochberg q within family A and within family B.
- **Sensitivity analyses**, for every tested outcome of families A and B:
  - random effects (DerSimonian–Laird on per-comparison log risk ratios, 0.5 added to every cell of a comparison with a zero cell), with τ², I² and the heterogeneity p;
  - Peto's odds ratio;
  - the **dominant preferred term**: the outcome's preferred term with the most participants affected over all comparisons, both arms together, analysed alone;
  - a **conservative variance**: the Greenland–Robins variance multiplied by the outcome's **overlap bound**, how far its count could overstate participants. In each comparison, at least as many participants had the outcome as had its most frequent preferred term (both arms together), so the bound is the outcome's count over all comparisons divided by the sum over comparisons of that largest term. It is capped at the **overcount factor**, the sum over every term of participants affected divided by participants with any serious adverse event, over all comparisons and both arms, which is as far as serious events as a whole are double counted. Both are computed blind to arm. The bound is 1 for an outcome of one preferred term;
  - leave one trial out (every comparison of that trial), reporting the range of risk ratios and the largest p;
  - selective GLP-1 receptor agonists only (`data/molecules.json`: tirzepatide, retatrutide, survodutide, cotadutide, mazdutide, efinopegdutide, pemvidutide and maridebart cafraglutide act on other receptors too);
  - type 2 diabetes, and overweight or obesity without diabetes, separately, with the z test of their difference.
- **Halves.** Trials are paired by size (participants in their comparisons, largest first, ties by NCT number), and one of each pair is assigned to half 1 by `numpy.random.default_rng(20261012)`; an unpaired last trial goes to half 1. Every family A and B outcome reports each half.
- **Robust** (families A and B): q < 0.05, and p < 0.05 with the conservative variance, under random effects, and with every single trial left out.

## Against the VA atlas (family A)

Each family A outcome falls in one category: **confirmed** (q < 0.05, in the VA direction), **opposite** (q < 0.05, against it), **excludes the VA estimate** (not significant, but the 95% interval excludes the VA hazard ratio), **inconclusive** (the interval holds both 1 and the VA hazard ratio), or **too few events**. A risk ratio from trials and a hazard ratio from a cohort are close for rare events over similar follow-up, which is how they are compared here.

An outcome is **informative** if a two-sided 5% test has at least 80% power to detect the VA hazard ratio, computed from the pooled number of participants affected and the allocation, blind to arm (`power.json`, computed before registration). The tally of categories is reported for all outcomes and for informative ones.

By `power.json`, 44 of the 49 family A outcomes have at least 10 participants affected and are tested: all but cannabis, stimulant and opioid use disorders, bulimia, and interstitial nephritis. Five are informative: bacterial infections (A18), sepsis (A20), pneumonia (A21), respiratory failure (A27) and acute pancreatitis (A49). Acute kidney injury (A17, 0.77) falls just short. All 13 family B outcomes are tested, as are 27 organ systems (family C) and 518 preferred terms (family D).

## Hypothesis-free scans (families C and D)

Family C is every organ system with at least 10 participants affected in all. Family D is every preferred term with at least 20. In each, the risk ratio is estimated in half 1 (discovery), with Benjamini–Hochberg q over the family. Family C uses the conservative variance with each organ system's overlap bound, since organ systems sum many terms; family D, whose items are single preferred terms, uses the Greenland–Robins variance. An item **replicates** if q < 0.05 in half 1 and p < 0.05 in half 2, in the same direction.

## Checks of the data

Two effects are established by the trials' own publications and labels: gallbladder and biliary disease (B01) and nausea and vomiting (A34) are more frequent on these drugs. They are positive controls: if either is not higher with q < 0.05, the paper says so, and treats every other estimate as suspect.

## What was seen before this plan was registered

- The record format, by reading one record (SELECT, NCT03574597), which showed the arm counts of its first two serious terms (abdominal adhesions, abdominal discomfort).
- Counts pooled over both arms: participants and participants affected per trial, per term, per organ system and per outcome, for the power calculations and thresholds. No count by arm was computed or looked at.
- The published results of the large trials and of meta-analyses of these drugs, as anyone in the field knows them, and the VA atlas, which set family A.
- The analysis code was written and tested with each comparison's arms swapped at random (`code/analyze.py --blind`). Its results under that swap say nothing about the drugs and were used only to check that the code runs. The blinded run also gave `power.json`, whose numbers don't depend on which arm is which.
- The screeners kept their working scripts in one shared scratch folder. One reported opening a display script another had written there (a script, not anyone's coding) before moving to a folder of its own. Each term coder had a folder of its own.

## Files

- `plan.md`, `screening_codebook.md`, `outcome_codebook.md`: this plan and the codebooks.
- `coding/`: both screenings, both term codings, the third reader's decisions with reasons, the terms as coded (`terms_input.json`), and the agreement.
- `data/`: what the analysis reads, as registered: `screening.json`, `term_coding.json`, `outcomes.json` (each outcome, its family, and the VA hazard ratio and interval its results state), `molecules.json`, `parameters.json`, and `search.json`.
- `ctgov_digests.json`: the SHA-256 of each of the 539 records, so the data analysed are those registered.
- `code/`: the analysis (`analyze.py`) and its command (`run`), with `requirements.txt` (Python 3.12, numpy 2.5.3, scipy 1.18.1).
- `power.json`: what the blinded run gave that doesn't depend on which arm is which.

## Deviations

Any departure from this plan will be listed in the bundle's `deviations.json` with its reason.