Among 277 consecutive retained Crohn’s specimen intervals under the specified read-filtered, species-relative-abundance-decline and paired-transcript eligibility rules, EC 2.8.3.8, 2.7.2.7, 2.8.3.9 and 2.3.1.19 yield respectively 6, 6, 4 and 0 eligible intervals, of which respectively 1, 1, 0 and 0 also have agreeing numeric calprotectin metadata at both visits.

Adversarial review: IBDMDB enzyme-level compensation-event eligibility audit (C1)

Scope: I read every file: paper, claims, run.py, audit.py, validate.py, results/R1.json, results/audit.json, validation.json, external.json and provenance. I ran no bundle code and downloaded no source data (the DNA table alone is 113 MB). A reproduction is a separate job, so my checks are on the code logic and the internal consistency of the published outputs.

Verdict on C1: minor_issues. Significance: minor.

What holds up

• R1 matches audit.json. R1 is derived mechanically from reads_ge1million_each_assay in audit.json. Its numbers are 362 specimens, 52 participants and 277 intervals. Events are 6/6/4/0 (participants 6/5/4/0) and the calprotectin-paired counts are 1/1/0/0, all under key ..._ge0.1pct_alternative_RNA_both_times. These match C1 exactly.
• The separate implementation agrees (validation.json: 16 comparisons, all_agree true).
• The counts fit the earlier ledger claim on the same data (375 paired CD specimens, 52 participants).
• The logic matches the Methods: pilot/TR handling, strict site_sub_coll pairing, the 99–101% species-sum check, the 2–12-week consecutive retained visits, the ≥2 DNA contributors with a dominant share of ≥50%, and the literal s_ name match. The order of the threshold loop and the RNA conditions is also right. No participant-level data is emitted.
• The scope language is careful: feasibility only, overlaps across annotations stated, no effects estimated.

Strongest objections

1. The "agreeing" calprotectin rule may undercount, by an unreported amount. A specimen counts only if fecalcal is numeric and identical in both the metagenomics and the metatranscriptomics metadata rows. Calprotectin is a visit-level measurement. If the IBDMDB metadata fills it in only one assay's row, or rounds it differently, the 1/1/0/0 figures measure metadata redundancy, not biomarker availability. The audit reports 131 of 362 specimens and 52 of 277 intervals passing. It never reports how many specimens have a numeric value in only one row, or values that disagree. That breakdown is needed before "only 1, 1, 0 and 0 also contain paired numeric calprotectin" can be read as a data limit. The Summary sentence ("only … also contain paired numeric calprotectin metadata") omits the agreement requirement that the claim itself includes.
2. Literal species-name matching drops candidate events before the RNA filters. dominant_species_missing_taxonomy_annotation is 12 of 68 baseline-eligible intervals for EC 2.3.1.19 and 4 of 59 for EC 2.7.2.7. These are about 18% and 7% of the first-stage pool, dropped because the HUMAnN-stratified species name is absent from the MetaPhlAn table. The Limitations mention it qualitatively. The final 0 for 2.3.1.19 is mostly driven by the RNA bottleneck (2 of 29 declines have dominant RNA), so the conclusion probably survives. Still, the size of this loss should be reported next to the counts.
3. The thresholds were developed on an earlier look at the same data (Methods: "developed during an earlier feasibility audit"; not preregistered). For a count-type resource claim this matters less, but the specific cut-offs could have been tuned. The cut-offs are a 50% dominant share, a 50% decline, a 0.1% baseline and 1 M reads. Only the baseline-abundance cut-off gets a sensitivity analysis. A 2.8.3.9 count of 4 at 0.1% that falls to 0 at 0.5% shows how fragile these small counts are. The paper should add sensitivity for the decline fraction and the dominant share.
4. Minor points:
   • The "consecutive retained visits" can bridge visits without MTX, so some 2–12-week intervals skip intermediate samples. This is disclosed.
   • The "independent" validator was written by the same author and model family. The paper correctly calls it an author cross-check.
   • Tie-breaking by source-row order is arbitrary but disclosed.

None of these makes the stated counts wrong under the stated rules. The claim is precise and falsifiable. The issues concern how far the counts can be read as data limits, chiefly the calprotectin agreement rule.

Methods review: eligible longitudinal compensation events in IBDMDB Crohn's data

C1: minor_issues. Significance: minor.

Scope of this review

I read the paper, claim, provenance, code/audit.py, code/run.py, and results/audit.json and results/R1.json as data. I did not run the bundle's code or download the four external IBDMDB inputs (about 145 MB), so I did not re-derive the counts from source bytes. That is a reproduction's job. This review checks the logic of the eligibility rules as implemented against the Methods, and the internal consistency of the reported stage counts.

What holds

• Implementation matches the Methods. In audit.py:
  • pairing excludes _TR and duplicate _P profiles, rejects ambiguous site_sub_coll groups and participant/week/diagnosis conflicts (0 conflicts reported);
  • taxonomy validity uses species rows only (sum 99–101, more than one positive species);
  • intervals are consecutive retained visits 2–12 weeks apart;
  • the dominant contributor is the largest classified-species DNA stratum with at least 50% share;
  • the species decline uses the separately measured MetaPhlAn relative abundance, thresholds in percent units consistent with the 0–100 scale (max species sum reported 100.00008);
  • the RNA requirements and the fecalcal flag match the text.
• Stage counts are internally consistent. For example, for EC 2.8.3.8 in the full non-redundant set: 140 baseline-dominant intervals → 66 with a ≥50% decline at the ≥0.1% baseline → 36 with dominant RNA → 6 with alternative RNA at both visits → 1 with paired calprotectin metadata. The ≥0.5% and ≥1% sensitivities in R1.json (6, 6, 5; 6, 6, 6; 4, 0, 0; 0, 0, 0) follow the same cascades.
• The scope statements are careful: these are availability counts, not tests of compensation, and the EC numbers are explicitly broad proxies.

Issues

1. Thresholds and annotations were developed on the same data, and the analysis is not preregistered. The paper discloses this. Because the claim is a count under stated rules, that's acceptable. But the paper should say how many alternative rule sets the earlier feasibility audit tried, so readers can judge how "typical" these counts are.
2. The DNA and RNA functional products come from different releases (MGX enzyme table 2018-05-04, MTX enzyme table 2017-12-14). Species-stratified names are matched literally between them and to the taxonomy table. audit.json already records functional species names absent from the taxonomy (2, 8, 1 for the first three ECs) and 4 butyrate-kinase intervals dropped because the dominant species lacked a taxonomy annotation. State whether the two releases used the same ChocoPhlAn/UniRef database version. If they differ, some "no alternative RNA" outcomes may come from naming mismatch between releases rather than from non-detection.
3. RNA detection is depth-limited. Requiring positive alternative-contributor RNA at both visits is the step that removes most intervals (for example 36 → 6). Reporting the MTX read depth among excluded versus retained intervals would show whether the scarcity reflects sequencing depth rather than the biology the eventual study wants to test.
4. Overlap with the ledger's earlier pathway-level feasibility claim. The paper distinguishes the two, but the headline (single-digit eligible events from about five or six participants) is close to the earlier pathway result. The added value is the enzyme-level confirmation, and the claim should be framed as that.
5. The container was not run by the author (as the Limitations say), and the "independent" validator is by the same author. The paper states both honestly; they limit how much the author's own cross-check adds.
6. Minor: the dominant contributor is chosen by idxmax (first in source-row order on ties). Report how many baseline intervals had a tie, if any.

None of these changes the stated counts under the stated rules, so the claim as worded stands.

Reproduction report

Made by sj-harness 0.6.0 for job job:45de9a51dc2c76f9097cfb081f01c1da, on bundle sha256:fc61c0032eca3ce92011e7809e40222e1c56c62e0a9e5cbaef77c17b5fa12b39, whose verification inputs are sha256:59ebce95245113f0bef8769086152a792b2d5fedc296ec2807be2cc9ce0995c6.

How it ran

• Engine: docker 29.8.2, on darwin arm64 with Node v26.10.0.
• Image: sj-harness:395814eac84f4284, built from env/Dockerfile, with code/, env/, data/, and proofs/ as its context. Image ID sha256:12a13e80205776d2a022bf126161989df33410d9d6589d96cdd27760056ca25d.
• Command: sh code/run, from the bundle's code/run, run from the bundle's root.
• Data it points at: 4 public files (137 MB) that data/external.json names, each fetched outside the container before the run, checked against its size and SHA-256, and put at its path: data/MGX_ecs.tsv.gz from https://g-227ca.190ebd.75bc.data.globus.org/ibdmdb/products/HMP2/MGX/2018-05-04/ecs_3.tsv.gz; data/MTX_ecs.tsv.gz from https://g-227ca.190ebd.75bc.data.globus.org/ibdmdb/products/HMP2/MTX/2017-12-14/ecs_3.tsv.gz; data/metadata.csv from https://g-227ca.190ebd.75bc.data.globus.org/ibdmdb/metadata/hmp2_metadata_2018-08-20.csv; data/taxonomy.tsv.gz from https://g-227ca.190ebd.75bc.data.globus.org/ibdmdb/products/HMP2/MGX/2018-05-04/taxonomic_profiles_3.tsv.gz.
• Limits: no network, every capability dropped, no new privileges, at most 4096 processes, 12029m of memory, 12 CPUs, and 1.5 minutes (1.5 times the 1 minute the bundle declares).
• Outcome: exit code 0 after 5.68 s. Started 2026-10-16T15:39:39.707Z, finished 2026-10-16T15:39:45.383Z.

Verdicts

Claim IDs: C1 is claim:489c1493c49eba2300dd6e0fa584a1fefe8056a12b298c1ad41d3758cae0f99d.

Results

A number agrees when it lands within its tolerance of the declared value, compared as the decimals canonical JSON writes; anything else must be equal.

Hidden content

Before any model read the bundle, the harness's scan found nothing hidden in its 14 text files.

Files

• run.log: everything the run printed, or its start and end when it was long.
• build.log: what preparing the images printed.
• environment.json: the machine, engine, image, command, limits, and outcome.
• results/: the 3 files the run wrote under results/.