# Each analysis as its paper describes it: the rules and how they were settled

Added in the third version, in answer to the second version's methods review, which asked how far the recorded departures move the estimates. Each association file's `described` field implements what follows; `code/lib/pipeline.R` (`described_version`) fits it on the paper's own cycles and stops if a qualifying departure is left unnamed, and `code/lib/summary.R` (`described_counts`) counts the results.

## When the rules were written

The rules under "What the described version is" and "Reading what the text describes" were written before any described version but one (row220's, the worked example) was fitted. The coding check had already fitted the text's version of many single departures as variants, so the effect of some departures was known when they were written. The points under "Settled after the first fits" were settled after the described versions were first fitted and checked, two of them after a mock review read the results; each is applied to every file alike. The reading rule's tie-break, the reading that changes the followed computation less, leans the described versions toward the published estimate, since the followed computation's open choices were settled partly by closeness to it.

## What the described version is

The paper version (the association as it is, fitted on the paper's own cycles) with every departure that meets all three of these put back the way the paper's text describes it, all together:

1. it bears on the headline estimate (`affects_headline = TRUE`);
2. the re-implementation follows it (`followed = TRUE`); a departure it does not follow, such as participants counted twice, which it counts once, or an outcome coded in reverse, which it codes as stated, is already computed as described;
3. its kind is `coding`, `sample`, `weighting`, or `model`. A `reporting` departure stays as it is: it records two of the paper's own accounts disagreeing, or a number mislabeled, not a computation the text describes.

Nothing else changes: choices the paper left unstated keep the decisions the file records for them.

## Reading what the text describes

- The text is the paper's Methods, and where the Methods say nothing on the point, its other description of the analysis (Results text, table notes, figure legends).
- Take the most literal reading of the words. Where two readings remain, take the one that changes the followed computation least (fewer variables or steps changed). Never choose a reading by the estimate it gives.
- If the text's version cannot be built from the public files (a variable NHANES doesn't have, a description with no literal reading), don't approximate it: list the departure under `not_applied` with the reason.
- The described version's analytic sample is the complete cases of its own model, as the paper version's is of its model, unless the text describes the sample otherwise.

## How it is judged

For all 40 associations (one with no qualifying departure contributes its paper version): whether the described estimate falls outside the published 95% confidence interval, that is, whether the analysis as described fails the registered reproduction criterion; and, among the associations it changes, how many lack the published sign with p below 0.05 as described, beside how many do as followed. The ratio of the described estimate to the published one, and to the followed one, is summarized by its median.

## Settled after the first fits

- **A departure where one of the paper's own accounts, or its own steps read literally, gives the computation the paper version follows** is a reporting departure, as row311's became in the second version: the paper describes its computation two ways, and there is nothing to put back. This was settled after the mock review's second round for eight departures that had been in `applies` with no change: row062's first, row087's second and third, row131's third, row218's third and fourth, row303's second, and row327's second.
- **Cutpoints** a paper prints are part of what its text describes and are kept; cutpoints the file computes from the sample (a `constants` function) are recomputed on the described version's data.
- **A described fit that does not converge**, or gives no finite estimate, has no estimate and, like a 2021-2023 model that can't be fitted under the plan, meets neither criterion. (No described fit now fails to converge.)
- **A named index or equation beside a printed formula that contradicts it**: where the name is what the paper version computes (rows 062, 218, 303, and 327), the departure is a reporting departure by the point above; where neither the name nor the printed formula is (row295), the reading that changes the followed computation less is taken, which is the printed formula. Rows 218, 303, and 327 had at first taken the printed formula literally (one of those fits did not converge), and were changed after the mock review's first round.

## The blind checks and how their disagreements were settled

Four agents wrote the described versions of the 31 files other than row220. Two more agents, who saw neither results nor the writers' notes, checked all 32 specifications against these rules. They found nothing to disagree with in 28, and five disagreements in the other four, which the lead agent settled as follows:

1. **row062, eGFR.** The writer read the printed Cockcroft-Gault formula literally, creatinine in mmol/L, giving values a thousand times too large. The checker read the CKD-EPI equation the Methods name. Settled for the checker, by the rule on named equations above: no change.
2. **row062, tertiles.** The checker would recompute the tertiles on the regrouped index. Settled for the writer: the paper prints its cutpoints (0.79 and 2.60), so they are kept.
3. **row096, weight.** The writer took the examination weight; the checker the fasting subsample's, which NCHS directs for fasting values and which the file's own record of the text's version uses. Settled for the checker.
4. **row295, METS-VF.** The writer read the printed formula with the log over the glucose term only and glucose in mg/dL; the checker with the log over the whole product, as its square bracket closes, and glucose in the units the paper version uses, since the paper states none. Settled for the checker.
5. **row315, sample.** The writer kept the paper version's requirement that told hypertension be known; the checker required instead the measured-or-told hypertension the described model uses. Settled for the checker.

Separately, row249's writer recomputed its quartile cutpoints; by the rule on printed cutpoints they were restored to the paper's (17.73, 24.52, 33.29).