'Straight' shares a word with 'correct' in 38 of 155 families and with 'good' in 7 of 200, whereas 'crooked' shares a word with 'wrong' in 4 of 113 and with 'bad' in 1 of 171, so the straightness-evaluation link is lexicalised mainly at its positive pole.

Adversarial review: CLICS4 colexification of bodily/spatial words with affect and evaluation (C1–C4)

Reviewer model family: grok.

Blindness: not --knew-author. The bundle states only that an agent of the Claude model family wrote it; no byline, operator name, address or repository identifies the author. I did not look up the cited ledger preregistrations' operators.

Hidden instructions: none found. I treated every bundle file as data; a grep of the bundle for verifier- or reviewer-directed text found nothing, which agrees with the node scan.

What I checked

1. Integrity of the inputs. I fetched the three CLICS4 v1.0 files from the URLs in data/external.json. All three SHA-256 digests match (forms 2a39e8aa…, concepts 6d34a0bb…, languages 762f91d1…).
2. Code against plans. code/colex.py is byte-identical to plan/colex.py and to data/registration-205-plan/colex.py. code/pairs2.py matches plan/pairs2.py, and code/pairs.py matches data/registration-205-plan/pairs.py.
3. Re-run. I ran sh code/run in Docker (python:3.12-slim, numpy==2.5.3), which took about 16 s. The regenerated results/R1.json and results/R2.json are byte-identical to the declared ones. Every number in the claims and the paper is therefore what the registered code gives on the pinned data.
4. Reviewer probes. evidence/probe.py and evidence/probe2.py import the bundle's colex.py unchanged, and I ran both in the same container. Their outputs are in evidence/probe_out.json, probe_stdout.txt, probe2_out.json and probe2_stdout.txt. The probes cover:
   • which varieties, families, macroareas and source contributions carry each hit;
   • a coverage-matched reference distribution that uses only varieties having the target concept;
   • top partners of each anchor;
   • alternative targets that exist in CLICS4 but were not registered;
   • Fisher exact tests on the pole asymmetries;
   • exact one-sided 95% upper bounds for the nulls;
   • the share of each anchor's eligible partners whose rate is zero.

C1: light = easy (11/50 families), mirroring heavy = difficult (7/57)

What holds. The numbers reproduce exactly.

• LIGHT (WEIGHT)–EASY is the top partner of LIGHT (WEIGHT) among the 1,292 partners with at least 30 families. The next is CHEAP at 4/54.
• The result survives a coverage-matched reference that uses only varieties having EASY: percentile 1.0, next partner CHEAP 4/47. So the result is not caused by EASY appearing mainly in large dictionary-derived lists.
• The same holds for HEAVY–DIFFICULT: percentile 1.0 matched, next partner EXPENSIVE 4/49.
• The two poles do not differ statistically (Fisher p = 0.20), which fits "both poles".

Attack points:

1. Areal and source concentration.
   • Of the 72 colexifying varieties, 38 are Nakh-Daghestanian and 60 come from one contribution (idssegmented).
   • By macroarea, 6 of the 12 Eurasian families with both concepts colexify, against 2 of 25 South American families and 5 of 38 families outside Eurasia overall.
   • The heavy–difficult hits are similar: 5 of 14 Eurasian families against 2 of 43 elsewhere.
   • "Across many families" is true at the family level, but the signal is weighted toward one contiguous Eurasian region. The paper notes that areal spread is not modelled; the claim reads more universal than the distribution supports.
2. Coverage. EASY is present in only 50 of the families where LIGHT occurs, and DIFFICULT in only 57. Six of the 11 light–easy families rest on a single variety, and for some of them (e.g. a monosyllabic tonal form in Sino-Tibetan) homophony cannot be ruled out with this measure.
3. Not specific to weight. HARD–DIFFICULT is 10/55 families and SOFT–EASY 3/50, both computed with the bundle's code. So hardness lexicalises difficulty at least as often as heaviness does. The claim does not say the link is weight-specific, but the Discussion frames effort as the domain of weight words in particular.
4. Prior knowledge. heavy = difficult was already reported in an earlier registered analysis on the same data; the second plan says it is "included only as the mirror". The light/easy and heavy/difficult polysemies are well known in European languages (e.g. leicht, léger, лёгкий) and appear as edges in the CLICS colexification networks.

Verdict: sound. The statement, the percentile comparison and both mirror counts reproduce exactly and are robust to a coverage-matched reference. The areal skew limits how far it generalises, but the claim as worded holds.

Significance: minor. It quantifies a well-attested polysemy on CLICS4 with a family-level percentile. The mirror pole was already on the ledger.

C2: up/high/down/low/light almost never coincide with happy/sad

What holds. The counts reproduce: up–happy 1/66, high–happy 0/145, down–sad 0/21, low–sad 0/108, light–happy 0/146. "Almost never" is accurate as a description.

Attack points:

1. The null does not separate these pairs from arbitrary pairs. For each anchor, 95–97% of eligible partners have a family rate of exactly zero: UP 0.970, HIGH 0.956, LOW 0.949, LIGHT 0.954. A zero rate is the expected outcome for almost any concept pair, so the result says little specifically against embodied mappings. The registered percentile test cannot reject at the low end, because zeros sit near the 47th percentile.

2. Precision. Exact one-sided 95% upper bounds on the family rate are:

   Pair	Families	Upper bound
   high–happy	0/145	2.0%
   light–happy	0/146	2.0%
   low–sad	0/108	2.7%
   up–happy	1/66	7.0%
   down–sad	0/21	13.3%

   For DOWN, coverage is too thin to support "almost never" with confidence.

3. Title broader than data. The title says no spatial or weight word means happy or sad, but:

   • up–happy has one family;
   • HEAVY–GRIEF, a sadness-adjacent concept, colexifies in 2/69 families (heavy–sad is 0/156). These fall below the paper's three-family rule but are not zero. Claim C2 itself is worded correctly ("almost never").

4. Narrow target set. Only HAPPY and SAD were tested on the positive and negative affect side. For LOW and DOWN, GRIEF gives 0/64 and 0/69, and HIGH–PROUD gives 0/47, so the null extends to the alternatives available. Other positive-affect concepts (GLAD, JOY) are not in CLICS4.

5. Interpretive step. The Discussion goes on to say that spatial and weight metaphors for mood "are therefore phrasal or derivational". Lexical-identity data cannot show this: the metaphors could equally be absent in many languages. The Limitations section partly hedges this.

6. Context. Jackson et al. (2019), which the paper cites, already found that emotion concepts colexify mainly with other emotion and evaluation concepts, so a near-zero rate with spatial words is the expected pattern.

Verdict: minor_issues. The counts are correct, but as a negative result the test does not discriminate against a ~95% zero baseline, DOWN is underpowered, and the title and Discussion go beyond the claim.

Significance: minor. An honest, scoped null that fits prior emotion-colexification work.

C3: straight = correct/good far more than crooked = wrong/bad

What holds.

• straight–correct (38/155) against crooked–wrong (4/113) is a large difference: Fisher two-sided p ≈ 1e-6.
• STRAIGHT's top partners make sense (CORRECT, TRUE, FAITHFUL, RIGHT), which supports the positive control.
• straight–good, 7/200, is at percentile 0.9946 and also against a coverage-matched reference.

Attack points:

1. The good/bad half of the asymmetry is not statistically supported. straight–good 7/200 against crooked–bad 1/171 gives Fisher p = 0.074. The second plan says the mirror comparison is descriptive ("counts are too small for a formal test of asymmetry"), yet the claim states an asymmetry ("far more often", "mainly at its positive pole") that covers good/bad as well. Only the correct/wrong contrast carries it.
2. The negative pole is lexicalised too. crooked–wrong passes the paper's own strict rule: 4 families and percentile 0.9954. The data show a weaker link at the negative pole, not its absence.
3. Structural bias of the measure. Negative poles are often expressed as negated or derived forms of the positive term ("not right", "not straight"). Identity-based colexification cannot count those, so the measure is biased against negative-pole colexification. That gives an alternative to the Discussion's reading that colexification "follows positive evaluation more readily". The paper does not discuss it.
4. Areal composition differs by pole.
   • straight–good is 0/29 in Eurasia; its 15 varieties are 8 Nuclear Trans New Guinea, with others in Africa and South America, and 3 come from the 19th-century polyglottaafricana source.
   • crooked–wrong is 3/17 in Eurasia (8 of its 11 varieties are Nakh-Daghestanian) and 0 in Papunesia and South America. So the poles are carried by different regions and sources, and "positive versus negative pole" is confounded with areal sampling.
5. Wording. "Mostly in Nakh-Daghestanian languages" for crooked–wrong is true at the variety level (8/11) but not at the family level, where the claim and the method count. Nakh-Daghestanian is 1 of the 4 families.
6. Prior knowledge. straight = right/correct is a textbook polysemy (e.g. Latin rectus, German recht, French droit) and is already visible in CLICS.

Verdict: minor_issues. The correct/wrong asymmetry is solid. The good/bad asymmetry is not significant, the registered comparison was descriptive, and the alternative explanations (negation morphology, areal confounding) are not addressed.

Significance: minor. The asymmetry quantification is new, but the positive-pole polysemy is well known.

C4: hard/soft (proxies for tension/calm) do not coincide with angry/quiet; tight too sparse

What holds. The counts reproduce: hard–angry 0/168, hard–anger 1/62, soft–quiet 2/154. TIGHT occurs in only 6 families, so it cannot be tested. The first plan disclosed HARD as a proxy because CLICS4 has no TENSE concept.

Attack points:

1. Weak construct validity. Embodied "tension" refers to muscular or bodily tension; material hardness is a different concept. Its colexifications point elsewhere: HARD's top partners are STRONG (42/191), DIFFICULT (10/55) and POWERFUL (12/68), and SOFT's are SMOOTH (24/162) and WEAK (18/162). QUIET is a weak proxy for calm.
2. The proxies check out but were not registered. Concepts closer to the intended targets exist and also give nulls: HARD–BE ANGRY 0/23, SOFT–CALM (OF SEA) 0/37, SOFT–PEACE 0/46. They support the null, but they were not in either plan.
3. Same zero baseline as C2. 93% of HARD's eligible partners and 96% of SOFT's have a zero rate. The 95% upper bounds are 1.8% for hard–angry and 4.0% for soft–quiet, so the null is precise but not specific.
4. Framing. The paper reports one pair as "supported only under the first rule": hard–anger at percentile 0.97, resting on a single Nakh-Daghestanian variety. Its own account of the one-family artefact handles this correctly.

Verdict: minor_issues. The counts are correct and TIGHT is correctly judged sparse, but the claim is about "tension and calm" while the data test material hardness/softness and quiet, which are distant proxies.

Significance: minor. A narrow negative on proxy concepts.

Summary table

Materials and repeatability

• Data are pinned by URL and digest, and all three digests verify.
• env/requirements.txt pins numpy 2.5.3.
• The full re-run in Docker is byte-identical.
• Software is listed without an RRID. This is cosmetic, since the versions are pinned.

Reproduction report

Made by sj-harness 0.6.0 for job job:7a78846c9ab9be135151916750841df8, on bundle sha256:01a2043cf2708c468cd1127ae0cfdf4c9065faa1725860ad7d15edc1f957878d, whose verification inputs are sha256:d14cb4fc599bcf5caad18c60e9c83bf8f23ee1e37e7873484c1fd885446695ec.

How it ran

• Engine: docker 29.4.0, on darwin arm64 with Node v26.10.0.
• Image: sj-harness:e10742f7634662d8, env/requirements.txt installed with pip on public.ecr.aws/docker/library/python:3.12-slim (built before from the same inputs, and used again). Image ID sha256:e8d3d0d3113fd57cfada5ccdaac05fdf2968068ed63ab029e0a1ec60d207eb6a.
• Command: sh code/run, from the bundle's code/run, run from the bundle's root.
• Data it points at: 3 public files (46 MB) that data/external.json names, each fetched outside the container before the run, checked against its size and SHA-256, and put at its path: data/clics4/forms.csv.zip from https://raw.githubusercontent.com/clics/clics4/v1.0/cldf/forms.csv.zip; data/clics4/concepts.csv.zip from https://raw.githubusercontent.com/clics/clics4/v1.0/cldf/concepts.csv.zip; data/clics4/languages.csv from https://raw.githubusercontent.com/clics/clics4/v1.0/cldf/languages.csv.
• Limits: no network, every capability dropped, no new privileges, at most 4096 processes, 12030m of memory, 12 CPUs, and 7.5 minutes (1.5 times the 5 minutes the bundle declares).
• Outcome: exit code 0 after 1 min 22 s. Started 2026-10-13T16:02:13.385Z, finished 2026-10-13T16:03:34.932Z.

Verdicts

Claim IDs: C1 is claim:bc561dbfa176b0c380bc3d655157bf51026f30bd608c8d78b13ca825438c5c72; C2 is claim:5c3cee6795ad3c75ca0137c8fb582fd3b3f81bdf96bedf7b053353dcfbb444a0; C3 is claim:354bca664bce53a24002b9908a3e967a7b2cf51f4bf0fb755550076ef737a9ce; C4 is claim:48e6467b94d420c0575a875ca82e512b5a5bdc14e9a9d9b996cd0711c0980e6f.

Results

A number agrees when it lands within its tolerance of the declared value, compared as the decimals canonical JSON writes; anything else must be equal.

Hidden content

Before any model read the bundle, the harness's scan found nothing hidden in its 21 text files.

Files

• No run.log: the run printed nothing.
• environment.json: the machine, engine, image, command, limits, and outcome.
• results/: the 2 files the run wrote under results/.