Auto-Research PipelineA durable, human-gated pipeline for computational research
Research run44d2d5b89d84· 8 September 2026

Are age differences in personality domain-general or nuance-specific? Testing whether Big Five domain scores capture the age-related information carried by their own items

CompleteArchived runPsychology · Modelled on Psychological Science

Paper

Domain Scores Discard Half the Age Signal in Their Own Items: A Loading-Conditioned Test of the Domain-General Model

Abstract

Do people differ with age on broad personality domains, or on the specific characteristics domain scores average over? We tested the domain-general (common-cause) model in archival responses from 2,401 Synthetic Aperture Personality Assessment participants who completed a 25-item Big Five inventory. If age relates to an item only through its domain factor, item–age correlations within a domain must be proportional to the items' loadings. That over-identifying restriction was rejected in all five domains — marginally for openness, χ2(4)=10.50\chi^2(4) = 10.50, p = .033 — and globally, χ2(20)=175.45\chi^2(20) = 175.45, p < .001. The residual profile replicated across random split halves (r = .788, 95% CI [.664, .887]), and out of sample the 25 items explained 8.4% of the variance in age against 3.7% for their domain scores, ΔR2=.047\Delta R^2 = .047, 95% CI [.028, .066].

§Ask about this paper

Grounded in this run

Questions are answered from this run's record — the paper, the executed analysis, the data provenance and the referee report. Nothing else is consulted.

This run is finished, so the recorded paper cannot be rewritten. Questions are still answered from its record.

Pipeline

1.Literature review

Attempt 3Done

Literature review complete (live:agent:claude-opus-5): 10 key papers, 18 grounded sources, 6 open gaps.

Summary

Personality psychology has two well-established descriptive facts about age. First, mean levels of the Big Five differ systematically across the life course - the "maturity principle" of rising conscientiousness, agreeableness and emotional stability documented meta-analytically by Roberts, Walton and Viechtbauer (2006) and mapped year by year in a 1.27-million-person web sample by Soto, John, Gosling and Potter (2011). Second, and less comfortably for the five-factor framework, the components of a single domain do not move together: Roberts et al. found social dominance and social vitality, both extraversion facets, moving in opposite directions.

A separate line of work has followed that crack down to the item level. Mottus, Kandler, Bleidorn, Riemann and McCrae (2017) argued that the trait hierarchy extends below facets to "nuances" - the specific variance of single items - and showed this residual variance has trait-like cross-rater agreement, stability and heritability rather than being measurement error. McCrae and Mottus (2019) generalised the point into a revised psychometrics in which item variance decomposes into domain, facet, nuance and bias components. The direct precedent for the present question is Mottus and Rozgonjuk (2021): in roughly 24,000 adults given a 300-item inventory, models predicting age achieved r = .28 from five domain scores, r = .44 from 30 facets and r = .65 from the raw items, and the unique variance of over a third of 240 NEO-PI items correlated with age. Hang and colleagues (2021) replicated this across N = 51,524 in several cultures and instruments, locating most of the unique age-sensitive information at the nuance level. The same ordering holds for life outcomes (Seeboth & Mottus, 2018; Stewart et al., 2022), prompting a programmatic call to move beyond the "Big Few" (Mottus et al., 2020).

Two things this literature has not done. It has never formally tested the domain-general model itself. The claim "items beat domains" is compatible with a purely domain-general world, because items differ in loading and reliability and an optimally weighted composite must beat an unweighted one. The common-cause model makes a sharper prediction - within a domain, item-age correlations should be proportional to the items' factor loadings - which is an over-identified restriction and therefore falsifiable. And the evidence comes almost entirely from long inventories; whether nuance-level age information survives in the 20-30 item short forms most researchers actually administer, where there are five items per domain and no facet scores at all, is untested. Olaru, Jankowsky and Allemand (2024) add a caution: age-varying item variances and loadings are themselves real, so a credible test must hold loadings fixed rather than assume equal ones.

Key papers (10)

  1. Development is in the details: Age differences in the Big Five domains, facets, and nuances. Mottus, R., & Rozgonjuk, D. - Journal of Personality and Social Psychology, 120(4), 1035-1048, 2021.
    The direct precedent. In ~24,000 internet respondents aged 18-50 with a 300-item inventory, age was predicted at r = .28 from domains, r = .44 from facets and r = .65 from items, and the unique variance of more than a third of 240 NEO-PI items correlated with age. Establishes the descriptive claim this study converts into a formal model test, and does so in a long inventory rather than a short form.
  2. Age differences in the personality hierarchy: A multi-sample replication study across the life span. Hang, Y., Soto, C., Speyer, L. G., Haring, L., Lee, B., Ostendorf, F., & Mottus, R. - Journal of Research in Personality, 93, 104121, 2021.
    Replicates Mottus and Rozgonjuk across N = 51,524 in several samples, age ranges, cultures and instruments: lower hierarchy levels, especially nuances, carry substantially more age-sensitive information, and most of the UNIQUE age-sensitive information sits at the nuance level. Shows the phenomenon is robust but still leaves the underlying model untested.
  3. Personality traits below facets: The consensual validity, longitudinal stability, heritability, and utility of personality nuances. Mottus, R., Kandler, C., Bleidorn, W., Riemann, R., & McCrae, R. R. - Journal of Personality and Social Psychology, 112(3), 474-490, 2017.
    Introduces the nuance construct and shows item-specific variance behaves like a trait (cross-rater agreement, rank-order stability, heritability, incremental utility) rather than like error. This is the theoretical warrant for treating within-domain item residuals as signal.
  4. What personality scales measure: A new psychometrics and its implications for theory and assessment. McCrae, R. R., & Mottus, R. - Current Directions in Psychological Science, 28(4), 415-420, 2019.
    States the measurement model at issue: classical 'score = trait + random error' is wrong, and item variance decomposes into domain, facet, nuance and systematic bias. Also flags evaluative and acquiescent bias as item-level variance that can masquerade as nuance signal - a caution this study must carry.
  5. Successful explanations start with accurate descriptions: Questionnaire items as personality markers for more accurate predictions. Seeboth, A., & Mottus, R. - European Journal of Personality, 32(3), 186-201, 2018.
    Shows the nuance-beats-domain pattern generalises beyond age to life outcomes, and reports the striking result that removing all Big Five variance from Big Five items barely changed their outcome prediction. Frames the general question of how much a domain score discards.
  6. Structural and inter-individual differentiation in personality traits across the adult lifespan. Olaru, G., Jankowsky, K., & Allemand, M. - European Journal of Personality, 38(2), 306-325, 2024.
    Examines loadings, inter-domain correlations and variances across ages 16-90 in three samples of 3,000+ and finds age variation in inter-individual differences at the nuance level but not at the domain or facet level. Supplies the main alternative explanation to guard against: age-varying loadings could in principle mimic nuance-specific age effects.
  7. Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies. Roberts, B. W., Walton, K. E., & Viechtbauer, W. - Psychological Bulletin, 132(1), 1-25, 2006.
    The canonical longitudinal evidence for the maturity principle from 92 samples, and the first clear signal of within-domain heterogeneity: the extraversion facets social dominance and social vitality change in opposite directions. Also the only non-cross-sectional anchor available, which matters for the age-differences-versus-change caveat.
  8. Age differences in personality traits from 10 to 65: Big Five domains and facets in a large cross-sectional sample. Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. - Journal of Personality and Social Psychology, 100(2), 330-348, 2011.
    Benchmark cross-sectional age curves from N = 1,267,218 web respondents aged 10-65, with domain-specific and non-monotonic trajectories and facets diverging within domains. Sets the expectation for the size and direction of domain-level age correlations in internet samples of this kind.
  9. Constraints on Generality (COG): A proposed addition to all empirical papers. Simons, D. J., Shoda, Y., & Lindsay, D. S. - Perspectives on Psychological Science, 12(6), 1123-1128, 2017.
    The reporting standard for bounding inferences to the sampled population - essential here, because every study in this literature, including the present one, uses a self-selected internet convenience sample.
  10. The new statistics: Why and how. Cumming, G. - Psychological Science, 25(1), 7-29, 2014.
    The target venue's estimation standard: effect sizes with confidence intervals and prespecification rather than bare significance testing. Dictates that the incremental-information claim be reported as an interval estimate, not a p value.

Methods in use

  • Large cross-sectional self-report internet samples with Likert items; reverse-keyed items recoded before scoring; complete-case or planned-missing designs.
  • Unit-weighted domain scores (mean of standardised items) as the default aggregation, with Cronbach's alpha or omega for reliability; factor scores used less often.
  • Level-by-level prediction contrasts: models predicting age (or an outcome) separately from domains, facets and items, compared on cross-validated accuracy - typically the correlation between predicted and observed age under k-fold CV with OLS or elastic net.
  • Nuance variance operationalised as the item residual after partialling out the domain or facet score, i.e. a regression residual rather than a latent variable.
  • Exploratory and confirmatory factor analysis of item correlation matrices to obtain domain loadings; principal-axis or ULS extraction for short scales.
  • Bootstrap and permutation inference where sampling distributions of derived quantities (profile correlations, differences in explained variance) are analytically awkward.
  • Split-sample and cross-country replication of item-level association profiles as an internal reliability check.
  • APA reporting standards at the target venue: exact p values, effect sizes with 95% confidence intervals, explicit sample-size justification, disclosure of all exclusions and analytic choices, and a Constraints on Generality statement.

Open gaps

  • The domain-general (common-cause) model has never been tested directly. It implies an over-identified restriction - within a domain, item-age correlations must be proportional to the items' factor loadings, r_i = lambda_i * beta_domain - which yields 4 degrees of freedom per five-item domain and can be tested with a minimum-distance statistic. Existing work reports prediction contrasts instead, which cannot distinguish nuance effects from optimal reweighting.
  • Differential item reliability and unequal loadings are a standing alternative explanation for 'items beat domains' results and are rarely modelled explicitly; a test that conditions on estimated loadings would close this loophole.
  • All the evidence comes from long inventories (300 items in Mottus & Rozgonjuk, 2021; long NEO- and BFI-family instruments in Hang et al., 2021). Whether nuance-level age information survives in the 20-30 item short forms that most researchers administer - five items per domain, no facet scores - is untested and is the practically consequential case.
  • The replicability of the nuance-level age profile has been assessed only across country samples and only 'at least moderately'; a within-sample split-half estimate of the reliability of the whole 25-item residual profile, with a confidence interval, would establish whether the heterogeneity is signal or sampling noise.
  • Whether the increment is practically consequential, not merely detectable, is unresolved. The right currency is a cross-validated increment in explained variance with a confidence interval, comparing item-level against domain-level predictors trained and tested out of sample.
  • Age-varying loadings and variances (Olaru et al., 2024) and item-level acquiescent or evaluative bias (McCrae & Mottus, 2019) are two mechanisms that could generate apparent nuance-age associations; neither is routinely ruled out, and a demographic covariate check plus an explicit loading-conditioned test would address them.
Grounded search sources (18) · mode: live:agent:claude-opus-5

2.Ideation

Attempt 2Done

3 candidate ideas generated; selected: “Are age differences in personality domain-general? A loading-conditioned test of the common-cause model in a 25-item Big Five short form”.

  1. Idea 1Selected

    Are age differences in personality domain-general? A loading-conditioned test of the common-cause model in a 25-item Big Five short form

    Hypothesis. The domain-general (common-cause) model of age differences is false. If age is related to an item only through that item's domain factor, then within each Big Five domain the item-age correlations must be proportional to the items' factor loadings: r(item_i, age) = lambda_i x beta_domain. H1 predicts that this over-identifying restriction is rejected in at least one domain (minimum-distance chi-square with 4 df per five-item domain, alpha = .05), that the resulting item-specific ('nuance') age residuals form a profile that replicates across random split halves of the sample, and that the 25 items predict age out of sample better than the five domain scores do. The hypothesis is refuted if the restriction cannot be rejected in any domain, or if the split-half profile correlation is near zero, or if the cross-validated increment in explained variance from items over domain scores is not reliably greater than zero.

    Dataset. Big Five Inventory: 25 personality items, SAPA project bfi_personality_items

    Test plan, novelty and risk

    Test plan. Variables: the 25 IPIP items (A1-A5, C1-C5, E1-E5, N1-N5, O1-O5; six-point Likert, 1 = very inaccurate to 6 = very accurate), age in years, gender (1 = male, 2 = female) as a covariate in a robustness check. Preprocessing: reverse-score the seven negatively keyed items (A1, C4, C5, E1, E2, O2, O5) as 7 - x; z-standardise every item and age within the analysis sample. Sample filters, fixed before analysis: retain participants aged 14-70 (the range in which the SAPA sample has usable density; drops implausible or near-empty tails) with complete responses on all 25 items; report the exact number excluded by each filter and the analysed n. Estimation. (1) Loadings: for each domain separately, extract a one-factor solution from the 5x5 item correlation matrix by iterated principal-axis factoring (squared multiple correlations as initial communalities) to obtain standardised loadings lambda_i. (2) Item-age correlations r_i for all 25 items. (3) Sampling covariance: a seeded nonparametric bootstrap over participants (B = 2,000) recomputing r and lambda in every resample, giving the 25x25 covariance V of r and percentile 95% CIs. (4) Primary test: per domain, the minimum-distance statistic Q_d = min_beta (r_d - lambda_d beta)' W (r_d - lambda_d beta) with W = V_d^-1, distributed chi-square(4) under the domain-general null, plus a bootstrap-calibrated p value from recentred resamples so that sampling error in lambda is not ignored; a global statistic summing across domains on 20 df. Report beta_d (the implied domain-factor-age correlation), each item's nuance residual e_i = r_i - lambda_i beta_d with a bootstrap 95% CI, and the count of within-domain sign reversals in r_i, which no attenuation account can produce. (5) Profile reliability: 500 seeded random split halves; recompute e in each half; correlate the two 25-value profiles; report the mean split-half r with a 95% percentile interval and the Spearman-Brown-corrected value. (6) Practical magnitude: 10-fold cross-validation with a fixed seed, ordinary least squares, predicting standardised age from (a) the five unit-weighted domain scores, (b) five per-domain Thurstone factor scores with weights estimated in the training folds only, and (c) the 25 items; report out-of-fold R-squared for each, the increment of (c) over (a) and over (b), a percentile bootstrap 95% CI for the increment, a paired test on the out-of-fold squared errors, and the spread across 20 alternative fold seeds. (7) Robustness: repeat the item-age correlations as partial correlations controlling for gender and report the correlation between the adjusted and unadjusted nuance profiles. Power: with the analysed n, report the sensitivity analysis - the smallest |r| detectable at alpha = .05 two-tailed with 80% power, and the noncentrality needed for 80% power at 4 df. Refutation: non-significant Q in every domain, a split-half profile correlation whose CI includes zero, or a cross-validated increment whose 95% CI includes zero.

    Novelty. Mottus and Rozgonjuk (2021) and Hang et al. (2021) established descriptively that items predict age better than domains, but a prediction contrast cannot distinguish genuine item-specific age effects from optimal reweighting of items that merely differ in loading and reliability - a domain-general world produces exactly that ordering. No study in this literature appears to have tested the common-cause restriction itself, which is over-identified for a five-item scale and therefore falsifiable. Second, all existing evidence comes from 240- to 300-item inventories; the practically consequential case is the 20-30 item short form that most researchers actually administer, where there are five items per domain and no facet scores at all, and where the nuance signal has every reason to be weaker. Third, the split-half reliability of the whole nuance profile has not been estimated within a sample; prior work reports only 'at least moderate' cross-country replication of single-item correlations.

    Expected outcome. The domain-general restriction is expected to be rejected in most domains, because published facet-level work already shows components of a domain moving in opposite directions with age. The genuinely uncertain quantities are the size of the cross-validated increment - it could easily be near zero in a 25-item instrument, which would be an informative null about practical consequence - and whether the nuance profile is reliable rather than sampling noise. Domain-level age correlations should land near |r| = .07-.20, consistent with Soto et al. (2011).

    Main risk. Three. (i) Cross-sectional age differences confound development with cohort and with differential self-selection into an online study, so nothing here licenses a claim about change; the paper must say age differences throughout. (ii) Item-level acquiescent and evaluative response bias also lives in item-specific variance and could inflate apparent nuance signal; the gender-adjusted robustness check and the sign-reversal count mitigate but do not eliminate this. (iii) The minimum-distance test treats the estimated loadings as fixed, which would make it anticonservative; the bootstrap recalibration that recomputes loadings in every resample is included precisely to address that, and any gap between the asymptotic and bootstrap p values must be reported.

  2. Idea 2

    Does the apparent five-factor structure of a short Big Five inventory depend on the respondent's age? A bootstrap test of age-varying congruence

    Hypothesis. Personality structure is age-differentiated: the five-factor solution recovered from younger respondents is not the same solution recovered from older respondents. Operationally, Tucker congruence coefficients between the five-factor loading matrices of a young subsample (age 14-24) and an older subsample (age 35-70) will fall below the conventional .95 'equivalence' threshold for at least one factor, and the mean absolute inter-factor correlation among the five domain scores will be reliably lower in the older group than in the younger group (differentiation). Refuted if all five congruence coefficients exceed .95 and the bootstrap 95% CI for the between-group difference in mean absolute inter-domain correlation contains zero.

    Dataset. Big Five Inventory: 25 personality items, SAPA project bfi_personality_items

    Test plan, novelty and risk

    Test plan. Variables: the 25 items, age. Reverse-score A1, C4, C5, E1, E2, O2, O5; complete cases on all 25 items; two age strata fixed in advance (14-24 and 35-70) so the groups are non-overlapping, with the exact n in each reported. In each stratum, extract five factors by principal-axis factoring with oblimin-style oblique rotation implemented via a target rotation to the a priori five-factor pattern, then compute Tucker's congruence coefficient factor by factor after Procrustes alignment. Separately compute unit-weighted domain scores, their 5x5 correlation matrix in each stratum, and the mean absolute off-diagonal correlation. Inference by seeded nonparametric bootstrap (B = 2,000) within strata: percentile 95% CIs on each congruence coefficient and on the between-stratum difference in mean absolute inter-domain correlation. Report Cronbach's alpha per domain per stratum, since a group difference in reliability is an alternative explanation for a difference in inter-domain correlation, and repeat the comparison on disattenuated correlations. Refutation: congruence above .95 for all five factors and a difference CI containing zero.

    Novelty. Olaru et al. (2024) examined structural differentiation across the adult lifespan in three large samples but found domain- and facet-level structure largely stable, with age variation confined to item-residual variance. Applying the same question to a 25-item public short form asks whether the very common practice of scoring a brief inventory identically across a wide age range is defensible, and whether the null result generalises to instruments with only five indicators per factor, where rotation is far less stable.

    Expected outcome. Congruence coefficients probably above .90 and plausibly above .95 for the better-defined factors (N, E) and lower for openness, whose alpha in this instrument is the weakest. A modest or null differentiation effect is the more likely outcome, which would be a useful constraint rather than a discovery.

    Main risk. Splitting into two strata roughly halves the effective n for each factor solution, and five-indicator factors rotate unstably, so congruence coefficients carry wide bootstrap intervals; the study could easily be underpowered to distinguish .93 from .96. The two strata also differ in composition (the young stratum is dominated by ages 17-22 in this sample), so any structural difference is confounded with education and with the recruitment channel.

  3. Idea 3

    Does acquiescent responding masquerade as personality? Testing whether the individual acquiescence factor in a short Big Five inventory carries demographic signal

    Hypothesis. The individual tendency to agree with items regardless of content is not measurement noise: an acquiescence index built from the seven negatively keyed items and their positively keyed counterparts will (a) show non-trivial internal consistency, (b) correlate reliably with age and education, and (c) statistically account for part of the observed age-domain correlations, such that correcting the domain scores for acquiescence changes at least one domain-age correlation by more than .03 in absolute value. Refuted if the acquiescence index has near-zero internal consistency, or if its correlations with age and education have 95% CIs containing zero, or if acquiescence correction leaves every domain-age correlation essentially unchanged.

    Dataset. Big Five Inventory: 25 personality items, SAPA project bfi_personality_items

    Test plan, novelty and risk

    Test plan. Variables: the 25 items in raw (unreversed) form, age, education (1-5 ordinal band, missing for 223 participants - report the analysed n with and without education), gender. Construct the acquiescence index as the person mean across the seven negatively keyed items and seven loading-matched positively keyed items in raw scoring, so that content largely cancels and agreement remains; estimate its internal consistency by splitting the balanced pairs. Correlate the index with age, education and gender, with bootstrap 95% CIs (B = 2,000, seeded). Then recompute all five domain scores after within-person ipsatisation (subtracting the person's acquiescence index from every item before reverse-keying) and compare each domain-age correlation before and after, with a bootstrap CI on the change. Report Cronbach's alpha of each domain before and after correction. Refutation: the CIs for the acquiescence-age and acquiescence-education correlations both contain zero and no domain-age correlation shifts by more than .03.

    Novelty. McCrae and Mottus (2019) identify systematic response bias as a component of item variance alongside domain, facet and nuance variance, but the applied literature on this widely used public dataset almost never quantifies it, and the keying of this particular 25-item form is unbalanced (7 negative, 18 positive), which is precisely the condition under which acquiescence contaminates domain scores rather than cancelling out.

    Expected outcome. A small but non-zero acquiescence-age correlation and shifts in domain-age correlations of roughly .01-.05, i.e. real but not large enough to overturn the maturity principle.

    Main risk. With only seven negatively keyed items and unbalanced keying, the acquiescence index is confounded with agreeableness and conscientiousness content, so a 'response style' correlation may simply be a trait correlation under another name - a confound that this dataset cannot fully resolve, since it lacks a purpose-built balanced-keying subset.

Ranking rationale

Idea 1 is ranked first because it converts a descriptive regularity that the field already accepts into a falsifiable model test with a clear refutation criterion, it is the only one of the three whose primary result cannot be produced by an artefact of unequal item reliability (the loading-conditioned restriction is exactly what disposes of that alternative), and its second outcome - the cross-validated increment - is genuinely uncertain in a 25-item instrument, so the study is informative whichever way it lands. It also uses the full analysed sample for every estimate rather than splitting it. Idea 2 asks a question of comparable theoretical interest but must halve the sample and rotate five-indicator factors, so its confidence intervals would be too wide to separate the hypotheses it is meant to separate; with the available n it risks an uninterpretable null. Idea 3 is well-posed and cheap, but the instrument's unbalanced keying means the acquiescence index is structurally confounded with agreeableness and conscientiousness content, and no analysis of these 25 items can break that confound, so a positive result would be ambiguous by construction.

3.Experimentation

Attempt 2Done

Experiment succeeded in 1 round (9.137s).

Succeeded1 round · exit 0 · 9.137s

Data provenance

Datasetbfi_personality_items

Big Five Inventory: 25 personality items, SAPA project

Source
R package 'psych' (William Revelle, Northwestern University), mirrored by the Rdatasets archive; data collected by the SAPA personality project
Citation
Items from Goldberg, L.R. (1999) A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In: Mervielde, I. et al. (eds) Personality Psychology in Europe, Vol. 7. Tilburg: Tilburg University Press, 7-28. Data collected in Spring 2010 by the SAPA project: Revelle, W., Wilt, J. and Rosenthal, A. (2010) Individual differences in cognition: new methods for examining the personality-cognition link. In: Gruszka, A., Matthews, G. and Szymura, B. (eds) Handbook of Individual Differences in Cognition. New York: Springer, ch. 2, 27-49. DOI: 10.1007/978-1-4419-1210-7_2. Distributed as bfi in Revelle, W. psych: Procedures for Psychological, Psychometric, and Personality Research. Northwestern University, Evanston, Illinois.
Licence
Redistributed in the R package 'psych' under GPL (>= 2) via the Rdatasets archive. No dataset-specific licence is published; the IPIP items are described as public domain but IPIP issues no formal licence text, and the SAPA project no longer publishes data terms.
File
https://vincentarelbundock.github.io/Rdatasets/csv/psych/bfi.csv
Observations
2,800
Variables (29)
subject_id, A1, A2, A3, A4, A5, C1, C2, C3, C4, C5, E1, E2, E3, E4, E5, N1, N2, N3, N4, N5, O1, O2, O3, O4, O5, gender, education, age
Units
Items A1-A5, C1-C5, E1-E5, N1-N5 and O1-O5 are six-point Likert responses from 1 (very inaccurate) to 6 (very accurate). Several items are negatively keyed and must be reversed before scoring: A1, C4, C5, E1, E2, O2 and O5. gender is 1 male, 2 female. education is a 1-5 ordinal band and age is in years. subject_id is the original SAPA respondent identifier.
Retrieved
8 Sept 2026, 15:48
Checksum
sha256 68ae71a96c2157b0…

Real, publicly available data — downloaded and checksum-verified before the analysis ran, never simulated.

Result (RESULT_JSON)

{
  "n_total": 2800,
  "n_excluded_age_filter": 43,
  "n_excluded_incomplete_items": 364,
  "n_excluded_either": 399,
  "n_analysed": 2401,
  "age_min_raw": 3,
  "age_max_raw": 86,
  "age_min": 14,
  "age_max": 68,
  "age_mean": 28.762182423990005,
  "age_sd": 10.759089300139408,
  "age_median": 26,
  "n_female": 1608,
  "n_male": 793,
  "pct_female": 66.97209496043315,
  "alpha": {
    "A": 0.7233,
    "C": 0.7425,
    "E": 0.7693,
    "N": 0.8183,
    "O": 0.6128
  },
  "r_domain_age": {
    "A": 0.1761,
    "C": 0.1149,
    "E": 0.0768,
    "N": -0.1101,
    "O": 0.0676
  },
  "beta_domain_factor_age": {
    "A": 0.1732,
    "C": 0.142,
    "E": 0.1044,
    "N": -0.1289,
    "O": 0.0896
  },
  "Q": {
    "A": 65.064,
    "C": 26.447,
    "E": 51.971,
    "N": 21.469,
    "O": 10.503
  },
  "p_chi2": {
    "A": 2.4947583841952725e-13,
    "C": 0.00002570537913475557,
    "E": 1.3990470977691859e-10,
    "N": 0.0002555765533720819,
    "O": 0.03275825229482816
  },
  "p_bootstrap": {
    "A": 0.0004997501249375312,
    "C": 0.0004997501249375312,
    "E": 0.0004997501249375312,
    "N": 0.0004997501249375312,
    "O": 0.031984007996002
  },
  "Q_global": 175.454,
  "df_global": 20,
  "p_chi2_global": 7.50780489773615e-27,
  "p_bootstrap_global": 0.0004997501249375312,
  "n_nuance_ci_excludes_zero": 9,
  "n_sign_reversals": 0,
  "n_item_r_ci_excludes_zero": 19,
  "rms_r_age": 0.0893031520037177,
  "rms_nuance": 0.04158277731538501,
  "nuance_share_of_rms": 0.46563616605216707,
  "splithalf_profile_r_mean": 0.78798666779214,
  "splithalf_profile_r_ci": [
    0.664397873200389,
    0.886588582161129
  ],
  "splithalf_spearman_brown": 0.8801863284698088,
  "cv_r2_domain": 0.03693,
  "cv_r2_factor": 0.02855,
  "cv_r2_item": 0.08396,
  "cv_r_domain": 0.1928,
  "cv_r_factor": 0.1699,
  "cv_r_item": 0.292,
  "delta_r2_item_vs_domain": 0.04702517693240227,
  "delta_r2_item_vs_factor": 0.055412305223590086,
  "delta_r2_ci": [
    0.028125596746881372,
    0.06593706181389113
  ],
  "delta_r2_ratio": 2.273258603179837,
  "cohen_f2_increment": 0.05133518186556607,
  "paired_t_sqerr": 4.628517857748758,
  "paired_t_df": 2400,
  "paired_p_sqerr": 0.000003879084280752195,
  "seed_sweep_mean_r2": {
    "domain": 0.03796630106913018,
    "factor": 0.029550589040354076,
    "item": 0.08526946016320856
  },
  "seed_sweep_sd_delta_item_domain": 0.001391990561645162,
  "gender_adj_profile_r": 0.9977239824738464,
  "gender_adj_Q": {
    "A": 63.571,
    "C": 25.98,
    "E": 49.929,
    "N": 28.82,
    "O": 10.004
  },
  "gender_adj_p_chi2": {
    "A": 5.145342359636008e-13,
    "C": 0.000031932798444963485,
    "E": 3.7354888764862505e-10,
    "N": 0.000008502911682075744,
    "O": 0.040361844043608874
  },
  "sensitivity_min_detectable_r": 0.05714862432193938,
  "chi2_crit_4df": 9.487729036781154,
  "ncp_for_80pct_power_4df": 11.94,
  "n_bootstrap": 2000,
  "n_splits": 500,
  "n_folds": 10,
  "items": [
    {
      "item": "A1",
      "domain": "A",
      "loading": 0.38248310583221895,
      "r_age": 0.1521507363406825,
      "r_ci": [
        0.11489186191765846,
        0.19067826167404645
      ],
      "predicted_r": 0.06625753197357333,
      "nuance": 0.08589320436710916,
      "nuance_ci": [
        0.04833801891133457,
        0.12352252727460453
      ],
      "r_age_gender_adj": 0.1474058961327228
    },
    {
      "item": "A2",
      "domain": "A",
      "loading": 0.6851121359466298,
      "r_age": 0.11011085251046132,
      "r_ci": [
        0.07367554427794044,
        0.14755100403824498
      ],
      "predicted_r": 0.11868194584489576,
      "nuance": -0.008571093334434454,
      "nuance_ci": [
        -0.04452912468685696,
        0.028639887305887863
      ],
      "r_age_gender_adj": 0.1041364091594152
    },
    {
      "item": "A3",
      "domain": "A",
      "loading": 0.7640318047927989,
      "r_age": 0.0675681308172175,
      "r_ci": [
        0.027267191557908372,
        0.10640962998519632
      ],
      "predicted_r": 0.13235319668495937,
      "nuance": -0.06478506586774187,
      "nuance_ci": [
        -0.10472035709815994,
        -0.026902378239766853
      ],
      "r_age_gender_adj": 0.062327113032284394
    },
    {
      "item": "A4",
      "domain": "A",
      "loading": 0.4978585660017461,
      "r_age": 0.1496650572447036,
      "r_ci": [
        0.1110907659386506,
        0.1834176822142868
      ],
      "predicted_r": 0.08624401797670031,
      "nuance": 0.0634210392680033,
      "nuance_ci": [
        0.024969887953019115,
        0.0976002533029782
      ],
      "r_age_gender_adj": 0.1458359638839201
    },
    {
      "item": "A5",
      "domain": "A",
      "loading": 0.6217502149702241,
      "r_age": 0.12715688091092658,
      "r_ci": [
        0.08862251119052444,
        0.16419599813664917
      ],
      "predicted_r": 0.10770576299920734,
      "nuance": 0.01945111791171925,
      "nuance_ci": [
        -0.01934055258148516,
        0.05595119807776315
      ],
      "r_age_gender_adj": 0.12376430221506036
    },
    {
      "item": "C1",
      "domain": "C",
      "loading": 0.5648612206550991,
      "r_age": 0.09135733574894209,
      "r_ci": [
        0.05031460522357081,
        0.13418423328182086
      ],
      "predicted_r": 0.08022922369160528,
      "nuance": 0.011128112057336798,
      "nuance_ci": [
        -0.03066939882293591,
        0.05279102026750633
      ],
      "r_age_gender_adj": 0.09099534091500384
    },
    {
      "item": "C2",
      "domain": "C",
      "loading": 0.622589683569662,
      "r_age": 0.018728042896807528,
      "r_ci": [
        -0.02611603059946652,
        0.05930463121609643
      ],
      "predicted_r": 0.08842860009626198,
      "nuance": -0.06970055719945445,
      "nuance_ci": [
        -0.1136675504010747,
        -0.028562080231942857
      ],
      "r_age_gender_adj": 0.016257526713132565
    },
    {
      "item": "C3",
      "domain": "C",
      "loading": 0.5697129706192632,
      "r_age": 0.06843946460363183,
      "r_ci": [
        0.02614286808658052,
        0.10746533941004097
      ],
      "predicted_r": 0.080918334784626,
      "nuance": -0.012478870180994169,
      "nuance_ci": [
        -0.05467078888072688,
        0.02614407891495999
      ],
      "r_age_gender_adj": 0.06641511374206063
    },
    {
      "item": "C4",
      "domain": "C",
      "loading": 0.6861470898863191,
      "r_age": 0.1371763478407454,
      "r_ci": [
        0.09717744571368944,
        0.1765961533636754
      ],
      "predicted_r": 0.09745588181109374,
      "nuance": 0.03972046602965165,
      "nuance_ci": [
        -0.0011663604382641769,
        0.07962056204304417
      ],
      "r_age_gender_adj": 0.13454300740764058
    },
    {
      "item": "C5",
      "domain": "C",
      "loading": 0.5826186693334383,
      "r_age": 0.08737960891100213,
      "r_ci": [
        0.047673535876953166,
        0.12800099746744598
      ],
      "predicted_r": 0.08275137651447814,
      "nuance": 0.00462823239652399,
      "nuance_ci": [
        -0.03470853940656387,
        0.045252693984806386
      ],
      "r_age_gender_adj": 0.08437247106806509
    },
    {
      "item": "E1",
      "domain": "E",
      "loading": 0.6002559409091514,
      "r_age": 0.05443179658598209,
      "r_ci": [
        0.015581847092176996,
        0.09200461123106989
      ],
      "predicted_r": 0.06266194891268481,
      "nuance": -0.008230152326702725,
      "nuance_ci": [
        -0.04789242917703817,
        0.02935657597720163
      ],
      "r_age_gender_adj": 0.04936125926595092
    },
    {
      "item": "E2",
      "domain": "E",
      "loading": 0.7373915116695895,
      "r_age": 0.12181870340370474,
      "r_ci": [
        0.0814899206453671,
        0.1605509233389147
      ],
      "predicted_r": 0.07697781243597983,
      "nuance": 0.04484089096772491,
      "nuance_ci": [
        0.004698905542689069,
        0.08346491841812972
      ],
      "r_age_gender_adj": 0.11978717344522118
    },
    {
      "item": "E3",
      "domain": "E",
      "loading": 0.5952717365418456,
      "r_age": -0.006718107909780987,
      "r_ci": [
        -0.04693562909121147,
        0.03421819309795376
      ],
      "predicted_r": 0.06214163759521338,
      "nuance": -0.06885974550499437,
      "nuance_ci": [
        -0.10829397534703993,
        -0.02762634894628628
      ],
      "r_age_gender_adj": -0.00855050768281186
    },
    {
      "item": "E4",
      "domain": "E",
      "loading": 0.6932616638623387,
      "r_age": 0.0053002894649654365,
      "r_ci": [
        -0.03762496875381127,
        0.04486623464052839
      ],
      "predicted_r": 0.07237100710451701,
      "nuance": -0.06707071763955157,
      "nuance_ci": [
        -0.11008319170983692,
        -0.027775494154115607
      ],
      "r_age_gender_adj": 0.0017240741660574472
    },
    {
      "item": "E5",
      "domain": "E",
      "loading": 0.5408110877159297,
      "r_age": 0.10202925643210708,
      "r_ci": [
        0.06342679773805075,
        0.1417263928034564
      ],
      "predicted_r": 0.05645637875493859,
      "nuance": 0.04557287767716848,
      "nuance_ci": [
        0.005813952777821495,
        0.08522996583664258
      ],
      "r_age_gender_adj": 0.0992968104125413
    },
    {
      "item": "N1",
      "domain": "N",
      "loading": 0.7923916362856417,
      "r_age": -0.0829298624097287,
      "r_ci": [
        -0.12092024304079987,
        -0.04175525670424702
      ],
      "predicted_r": -0.10214896847474542,
      "nuance": 0.01921910606501671,
      "nuance_ci": [
        -0.018706942290312896,
        0.06034649175734547
      ],
      "r_age_gender_adj": -0.08496618203950569
    },
    {
      "item": "N2",
      "domain": "N",
      "loading": 0.7614477334414338,
      "r_age": -0.09974446445461949,
      "r_ci": [
        -0.1437102606714482,
        -0.056805692745060594
      ],
      "predicted_r": -0.09815992112571569,
      "nuance": -0.0015845433289038149,
      "nuance_ci": [
        -0.045507202635968205,
        0.04139245369700271
      ],
      "r_age_gender_adj": -0.10475862269890882
    },
    {
      "item": "N3",
      "domain": "N",
      "loading": 0.7645657279375294,
      "r_age": -0.10759786012753011,
      "r_ci": [
        -0.15041487392523298,
        -0.06568184272586668
      ],
      "predicted_r": -0.0985618687320522,
      "nuance": -0.00903599139547792,
      "nuance_ci": [
        -0.05114534636943804,
        0.032626132619463316
      ],
      "r_age_gender_adj": -0.1138252633673117
    },
    {
      "item": "N4",
      "domain": "N",
      "loading": 0.5975849277028593,
      "r_age": -0.025369008388681693,
      "r_ci": [
        -0.06556330151227326,
        0.02129852411964935
      ],
      "predicted_r": -0.07703600233218225,
      "nuance": 0.05166699394350055,
      "nuance_ci": [
        0.011424066221448962,
        0.09844024604973103
      ],
      "r_age_gender_adj": -0.025300062651486172
    },
    {
      "item": "N5",
      "domain": "N",
      "loading": 0.5333044329119532,
      "r_age": -0.10344303666219296,
      "r_ci": [
        -0.14274716863595727,
        -0.06520209404177432
      ],
      "predicted_r": -0.06874946075948658,
      "nuance": -0.03469357590270637,
      "nuance_ci": [
        -0.07347248957709987,
        0.003130707218330445
      ],
      "r_age_gender_adj": -0.11517456356353203
    },
    {
      "item": "O1",
      "domain": "O",
      "loading": 0.5382306268013634,
      "r_age": 0.04554958719378937,
      "r_ci": [
        0.006261158417874538,
        0.0846377313436954
      ],
      "predicted_r": 0.04823923923789037,
      "nuance": -0.0026896520441010013,
      "nuance_ci": [
        -0.041861441605594706,
        0.036794552687530215
      ],
      "r_age_gender_adj": 0.050627171733785725
    },
    {
      "item": "O2",
      "domain": "O",
      "loading": 0.4789056760229611,
      "r_age": 0.03197512950570044,
      "r_ci": [
        -0.007902514135078387,
        0.06985932951701758
      ],
      "predicted_r": 0.04292220532924292,
      "nuance": -0.010947075823542485,
      "nuance_ci": [
        -0.05069782387558767,
        0.0278467406759941
      ],
      "r_age_gender_adj": 0.03392264300066035
    },
    {
      "item": "O3",
      "domain": "O",
      "loading": 0.6369063090000574,
      "r_age": 0.04188855312672712,
      "r_ci": [
        0.0007406890237166756,
        0.08079167923427276
      ],
      "predicted_r": 0.05708310579530491,
      "nuance": -0.015194552668577783,
      "nuance_ci": [
        -0.057398596513546056,
        0.02478444559387904
      ],
      "r_age_gender_adj": 0.04390675962496845
    },
    {
      "item": "O4",
      "domain": "O",
      "loading": 0.27841835684151006,
      "r_age": 0.0014888375884181688,
      "r_ci": [
        -0.036520338682968646,
        0.04083549519143276
      ],
      "predicted_r": 0.024953410406454993,
      "nuance": -0.023464572818036825,
      "nuance_ci": [
        -0.06166527027362892,
        0.016610123385975558
      ],
      "r_age_gender_adj": 0.0026754178211335026
    },
    {
      "item": "O5",
      "domain": "O",
      "loading": 0.5383276217045065,
      "r_age": 0.09081405533153113,
      "r_ci": [
        0.04838660274415922,
        0.1289746863217161
      ],
      "predicted_r": 0.04824793246362779,
      "nuance": 0.04256612286790332,
      "nuance_ci": [
        -0.0006440117421527098,
        0.08031645467307706
      ],
      "r_age_gender_adj": 0.09261143248554828
    }
  ],
  "hypothesis_supported": true
}
Experiment script · experiment_v1.py
"""Loading-conditioned test of the domain-general model of age differences in the Big Five.

Real data: 25 IPIP items + demographics from the SAPA project, loaded via `dataset.load()`.
Domain-general (common-cause) null: age relates to an item only through its domain factor, so
within a domain r(item_i, age) = lambda_i * beta_domain.  That is an over-identified restriction
(4 df per five-item domain) and is tested with a minimum-distance statistic.
"""
import json
import numpy as np
from scipy import stats as st
from dataset import load

DOMAINS = {"A": ["A1", "A2", "A3", "A4", "A5"], "C": ["C1", "C2", "C3", "C4", "C5"],
           "E": ["E1", "E2", "E3", "E4", "E5"], "N": ["N1", "N2", "N3", "N4", "N5"],
           "O": ["O1", "O2", "O3", "O4", "O5"]}
DOM_ORDER = ["A", "C", "E", "N", "O"]
ITEMS = [i for d in DOM_ORDER for i in DOMAINS[d]]
REVERSED = ["A1", "C4", "C5", "E1", "E2", "O2", "O5"]
AGE_LO, AGE_HI = 14, 70
B_BOOT, N_SPLITS, B_DELTA, K_FOLDS = 2000, 500, 4000, 10
SEED_BOOT, SEED_SPLIT, SEED_CV, SEED_DELTA = 20260908, 11, 7, 99

df = load()
print("loaded", df.shape, "columns ok:", all(c in df.columns for c in ITEMS + ["age", "gender"]))
n_total = int(len(df))
age_min_raw, age_max_raw = float(df["age"].min()), float(df["age"].max())
for c in REVERSED:                                    # 6-point Likert -> reverse as 7 - x
    df[c] = 7.0 - df[c].astype(float)

m_age = (df["age"] >= AGE_LO) & (df["age"] <= AGE_HI)
m_complete = df[ITEMS].notna().all(axis=1)
n_drop_age = int((~m_age).sum())
n_drop_incomplete = int((~m_complete).sum())
n_drop_either = int((~(m_age & m_complete)).sum())
d = df[m_age & m_complete].reset_index(drop=True)
N = int(len(d))
print("analysed n =", N, "| dropped by age filter", n_drop_age, "| dropped incomplete", n_drop_incomplete)

X = d[ITEMS].to_numpy(float)
age_raw = d["age"].to_numpy(float)
gender = d["gender"].to_numpy(float)
Z = (X - X.mean(0)) / X.std(0, ddof=1)
a = (age_raw - age_raw.mean()) / age_raw.std(ddof=1)

def paf(R, iters=60):
    """One-factor principal-axis loadings for a small correlation matrix."""
    smc = 1.0 - 1.0 / np.diag(np.linalg.pinv(R))
    h = np.clip(smc, 0.05, 0.99)
    lam = np.sqrt(h)
    for _ in range(iters):
        Rr = R.copy()
        np.fill_diagonal(Rr, h)
        w, v = np.linalg.eigh(Rr)
        lam = v[:, -1] * np.sqrt(max(w[-1], 1e-9))
        if lam.sum() < 0:
            lam = -lam
        hn = np.clip(lam ** 2, 0.01, 0.99)
        if np.max(np.abs(hn - h)) < 1e-9:
            h = hn
            break
        h = hn
    return lam

def item_stats(Zm, am):
    """Item-age correlations and per-domain one-factor loadings."""
    n = Zm.shape[0]
    zz = (Zm - Zm.mean(0)) / Zm.std(0, ddof=1)
    aa = (am - am.mean()) / am.std(ddof=1)
    r = (zz * aa[:, None]).sum(0) / (n - 1)
    R = np.corrcoef(zz, rowvar=False)
    lam = np.concatenate([paf(R[k * 5:k * 5 + 5, k * 5:k * 5 + 5]) for k in range(5)])
    return r, lam

def min_distance(r, lam, V):
    """Per-domain minimum-distance fit of r_d = lambda_d * beta_d."""
    Q, beta, e = {}, {}, np.zeros(25)
    for k, dm in enumerate(DOM_ORDER):
        sl = slice(k * 5, k * 5 + 5)
        rv, lv = r[sl], lam[sl]
        W = np.linalg.inv(V[sl, sl])
        b = float((lv @ W @ rv) / (lv @ W @ lv))
        res = rv - lv * b
        Q[dm] = float(res @ W @ res)
        beta[dm] = b
        e[sl] = res
    return Q, beta, e

def bootstrap(Zm, am, seed):
    rng = np.random.default_rng(seed)
    n = Zm.shape[0]
    rb = np.empty((B_BOOT, 25))
    lb = np.empty((B_BOOT, 25))
    for b in range(B_BOOT):
        idx = rng.integers(0, n, n)
        rb[b], lb[b] = item_stats(Zm[idx], am[idx])
    return rb, lb

# ---- primary: item-age correlations, loadings, minimum-distance test -----------------------
r_hat, lam_hat = item_stats(Z, a)
rb, lb = bootstrap(Z, a, SEED_BOOT)
V = np.cov(rb, rowvar=False)
Q_hat, beta_hat, e_hat = min_distance(r_hat, lam_hat, V)
Q_global = float(sum(Q_hat.values()))

# bootstrap calibration: recentre each resample on the fitted domain-general values
target = np.concatenate([lam_hat[k * 5:k * 5 + 5] * beta_hat[DOM_ORDER[k]] for k in range(5)])
Qb_dom = {dm: np.empty(B_BOOT) for dm in DOM_ORDER}
Qb_glob = np.empty(B_BOOT)
for b in range(B_BOOT):
    Qd, _, _ = min_distance(rb[b] - r_hat + target, lb[b], V)
    for dm in DOM_ORDER:
        Qb_dom[dm][b] = Qd[dm]
    Qb_glob[b] = sum(Qd.values())
p_boot = {dm: float((np.sum(Qb_dom[dm] >= Q_hat[dm]) + 1) / (B_BOOT + 1)) for dm in DOM_ORDER}
p_boot_global = float((np.sum(Qb_glob >= Q_global) + 1) / (B_BOOT + 1))

eb = rb - (lb * np.repeat([[beta_hat[dm] for dm in DOM_ORDER]], 5, axis=0).T.ravel())
ci_r = np.percentile(rb, [2.5, 97.5], axis=0)
ci_e = np.percentile(eb, [2.5, 97.5], axis=0)
print("global Q = %.2f on 20 df" % Q_global)

# ---- split-half reliability of the nuance profile -----------------------------------------
rs = np.random.default_rng(SEED_SPLIT)
prof_r = np.empty(N_SPLITS)
for s in range(N_SPLITS):
    perm = rs.permutation(N)
    h1, h2 = perm[:N // 2], perm[N // 2:]
    r1, l1 = item_stats(Z[h1], a[h1])
    r2, l2 = item_stats(Z[h2], a[h2])
    _, _, e1 = min_distance(r1, l1, V)
    _, _, e2 = min_distance(r2, l2, V)
    prof_r[s] = np.corrcoef(e1, e2)[0, 1]
sb = 2 * prof_r / (1 + prof_r)

# ---- cross-validated prediction of age ----------------------------------------------------
def thurstone(Ztr, Zte):
    Str, Ste = [], []
    for k in range(5):
        sl = slice(k * 5, k * 5 + 5)
        R = np.corrcoef(Ztr[:, sl], rowvar=False)
        w = np.linalg.solve(R, paf(R))
        Str.append(Ztr[:, sl] @ w)
        Ste.append(Zte[:, sl] @ w)
    return np.column_stack(Str), np.column_stack(Ste)

def cv_predictions(seed):
    rg = np.random.default_rng(seed)
    idx = rg.permutation(N)
    out = {k: np.zeros(N) for k in ("domain", "factor", "item")}
    for f in np.array_split(idx, K_FOLDS):
        tr = np.setdiff1d(idx, f)
        Dtr = np.column_stack([Z[tr][:, j * 5:j * 5 + 5].mean(1) for j in range(5)])
        Dte = np.column_stack([Z[f][:, j * 5:j * 5 + 5].mean(1) for j in range(5)])
        Ftr, Fte = thurstone(Z[tr], Z[f])
        for nm, (Xtr, Xte) in {"domain": (Dtr, Dte), "factor": (Ftr, Fte), "item": (Z[tr], Z[f])}.items():
            A = np.column_stack([np.ones(len(tr)), Xtr])
            coef, *_ = np.linalg.lstsq(A, a[tr], rcond=None)
            out[nm][f] = np.column_stack([np.ones(len(f)), Xte]) @ coef
    return out

pred = cv_predictions(SEED_CV)
sst = float(((a - a.mean()) ** 2).sum())
R2 = {k: float(1 - ((a - v) ** 2).sum() / sst) for k, v in pred.items()}
r_pred = {k: float(np.corrcoef(a, v)[0, 1]) for k, v in pred.items()}
d_item_dom = R2["item"] - R2["domain"]
d_item_fac = R2["item"] - R2["factor"]
sq_dom, sq_item = (a - pred["domain"]) ** 2, (a - pred["item"]) ** 2
tt = st.ttest_1samp(sq_dom - sq_item, 0.0)
rd = np.random.default_rng(SEED_DELTA)
dboot = np.empty(B_DELTA)
for b in range(B_DELTA):
    i = rd.integers(0, N, N)
    ss = float(((a[i] - a[i].mean()) ** 2).sum())
    dboot[b] = (1 - (sq_item[i]).sum() / ss) - (1 - (sq_dom[i]).sum() / ss)
seed_sweep = np.array([[1 - ((a - v) ** 2).sum() / sst for v in cv_predictions(100 + s).values()] for s in range(20)])

# ---- robustness: partial out gender -------------------------------------------------------
gz = (gender - gender.mean()) / gender.std(ddof=1)
Zg = Z - np.outer(gz, (gz @ Z) / (gz @ gz))
ag = a - gz * (gz @ a) / (gz @ gz)
r_g, lam_g = item_stats(Zg, ag)
rbg, _ = bootstrap(Zg, ag, SEED_BOOT + 1)
Vg = np.cov(rbg, rowvar=False)
Q_g, beta_g, e_g = min_distance(r_g, lam_g, Vg)

# ---- descriptives, power ------------------------------------------------------------------
alpha_rel, r_dom_age = {}, {}
for k, dm in enumerate(DOM_ORDER):
    Zi = Z[:, k * 5:k * 5 + 5]
    alpha_rel[dm] = float(5 / 4 * (1 - Zi.var(0, ddof=1).sum() / Zi.sum(1).var(ddof=1)))
    r_dom_age[dm] = float(np.corrcoef(Zi.mean(1), a)[0, 1])
z_crit = st.norm.ppf(0.975) + st.norm.ppf(0.80)
min_r = float(np.tanh(z_crit / np.sqrt(N - 3)))
crit4 = float(st.chi2.ppf(0.95, 4))
ncp80 = float(next(x for x in np.arange(0.0, 40.0, 0.01) if st.ncx2.sf(crit4, 4, x) >= 0.80))

n_sig_e = int(np.sum((ci_e[0] > 0) | (ci_e[1] < 0)))
reversals = 0
for k, dm in enumerate(DOM_ORDER):
    sl = slice(k * 5, k * 5 + 5)
    opp = np.sign(r_hat[sl]) == -np.sign(beta_hat[dm])
    excl0 = (ci_r[0][sl] > 0) | (ci_r[1][sl] < 0)
    reversals += int(np.sum(opp & excl0))

items_out = []
for j, name in enumerate(ITEMS):
    items_out.append({"item": name, "domain": name[0], "loading": float(lam_hat[j]),
                      "r_age": float(r_hat[j]), "r_ci": [float(ci_r[0][j]), float(ci_r[1][j])],
                      "predicted_r": float(lam_hat[j] * beta_hat[name[0]]),
                      "nuance": float(e_hat[j]), "nuance_ci": [float(ci_e[0][j]), float(ci_e[1][j])],
                      "r_age_gender_adj": float(r_g[j])})

hypothesis_supported = bool(
    min(p_boot.values()) < 0.05
    and np.percentile(prof_r, 2.5) > 0
    and np.percentile(dboot, 2.5) > 0)

result = {
    "n_total": n_total, "n_excluded_age_filter": n_drop_age,
    "n_excluded_incomplete_items": n_drop_incomplete, "n_excluded_either": n_drop_either,
    "n_analysed": N, "age_min_raw": age_min_raw, "age_max_raw": age_max_raw,
    "age_min": float(age_raw.min()), "age_max": float(age_raw.max()),
    "age_mean": float(age_raw.mean()), "age_sd": float(age_raw.std(ddof=1)),
    "age_median": float(np.median(age_raw)),
    "n_female": int((gender == 2).sum()), "n_male": int((gender == 1).sum()),
    "pct_female": float(100 * (gender == 2).mean()),
    "alpha": {k: round(v, 4) for k, v in alpha_rel.items()},
    "r_domain_age": {k: round(v, 4) for k, v in r_dom_age.items()},
    "beta_domain_factor_age": {k: round(v, 4) for k, v in beta_hat.items()},
    "Q": {k: round(v, 3) for k, v in Q_hat.items()},
    "p_chi2": {k: float(st.chi2.sf(v, 4)) for k, v in Q_hat.items()},
    "p_bootstrap": p_boot, "Q_global": round(Q_global, 3), "df_global": 20,
    "p_chi2_global": float(st.chi2.sf(Q_global, 20)), "p_bootstrap_global": p_boot_global,
    "n_nuance_ci_excludes_zero": n_sig_e, "n_sign_reversals": reversals,
    "n_item_r_ci_excludes_zero": int(np.sum((ci_r[0] > 0) | (ci_r[1] < 0))),
    "rms_r_age": float(np.sqrt(np.mean(r_hat ** 2))), "rms_nuance": float(np.sqrt(np.mean(e_hat ** 2))),
    "nuance_share_of_rms": float(np.sqrt(np.mean(e_hat ** 2)) / np.sqrt(np.mean(r_hat ** 2))),
    "splithalf_profile_r_mean": float(prof_r.mean()),
    "splithalf_profile_r_ci": [float(np.percentile(prof_r, 2.5)), float(np.percentile(prof_r, 97.5))],
    "splithalf_spearman_brown": float(sb.mean()),
    "cv_r2_domain": round(R2["domain"], 5), "cv_r2_factor": round(R2["factor"], 5),
    "cv_r2_item": round(R2["item"], 5),
    "cv_r_domain": round(r_pred["domain"], 4), "cv_r_factor": round(r_pred["factor"], 4),
    "cv_r_item": round(r_pred["item"], 4),
    "delta_r2_item_vs_domain": float(d_item_dom), "delta_r2_item_vs_factor": float(d_item_fac),
    "delta_r2_ci": [float(np.percentile(dboot, 2.5)), float(np.percentile(dboot, 97.5))],
    "delta_r2_ratio": float(R2["item"] / R2["domain"]),
    "cohen_f2_increment": float(d_item_dom / (1 - R2["item"])),
    "paired_t_sqerr": float(tt.statistic), "paired_t_df": int(N - 1), "paired_p_sqerr": float(tt.pvalue),
    "seed_sweep_mean_r2": {"domain": float(seed_sweep[:, 0].mean()), "factor": float(seed_sweep[:, 1].mean()),
                           "item": float(seed_sweep[:, 2].mean())},
    "seed_sweep_sd_delta_item_domain": float((seed_sweep[:, 2] - seed_sweep[:, 0]).std(ddof=1)),
    "gender_adj_profile_r": float(np.corrcoef(e_hat, e_g)[0, 1]),
    "gender_adj_Q": {k: round(v, 3) for k, v in Q_g.items()},
    "gender_adj_p_chi2": {k: float(st.chi2.sf(v, 4)) for k, v in Q_g.items()},
    "sensitivity_min_detectable_r": min_r, "chi2_crit_4df": crit4, "ncp_for_80pct_power_4df": ncp80,
    "n_bootstrap": B_BOOT, "n_splits": N_SPLITS, "n_folds": K_FOLDS,
    "items": items_out,
    "hypothesis_supported": hypothesis_supported,
}
print("RESULT_JSON: " + json.dumps(result))
Listing. experiment_v1.py

4.Analysis

Attempt 2Done

Analysis complete: experiment succeeded; hypothesis supported; 13 key numbers.

Hypothesis supportedExperiment succeeded

Results and analysis

The analysis ran to completion on the first attempt (exit code 0, no timeout) against the real SAPA Big Five item data, and returned a complete set of finite estimates. The hypothesis was supported on all three of its pre-specified criteria.

Sample and exclusions. Of the 2,800 archived respondents, 43 fell outside the pre-specified age window of 14-70 years (the raw file spans ages 3 to 86, with single respondents at the extremes) and 364 had at least one missing item response; 8 respondents met both exclusions, so 399 were excluded in total and n = 2,401 were analysed. In the analysed sample age ranged from 14 to 68 years (M = 28.76, SD = 10.76, Mdn = 26), and 1,608 participants (66.97%) were women and 793 were men. The five unit-weighted domain scores had Cronbach's alphas of .72 (agreeableness), .74 (conscientiousness), .77 (extraversion), .82 (neuroticism) and .61 (openness), the last below the conventional .70 threshold, which matters for interpreting the openness results.

Domain-level age differences replicate the literature. Correlations between age and the unit-weighted domain scores were r = .176 for agreeableness, .115 for conscientiousness, .077 for extraversion, -.110 for neuroticism and .068 for openness. Every one is in the direction and the magnitude range the maturity-principle literature documents for cross-sectional internet samples (|r| = .07-.18), so the data behave as expected before the critical test is applied.

Primary test: the domain-general restriction fails in all five domains. Under the common-cause model an item can be related to age only through its domain factor, so within a domain the item-age correlations must be proportional to the items' loadings. Fitting r_i = lambda_i x beta_d by minimum distance, with the 25 x 25 sampling covariance of the item-age correlations estimated by a seeded nonparametric bootstrap over participants (B = 2,000), the implied factor-age correlations were beta = .173 (A), .142 (C), .104 (E), -.129 (N) and .090 (O). The over-identifying restriction was rejected in every domain: agreeableness, chi-square(4) = 65.06, p < .001; extraversion, chi-square(4) = 51.97, p < .001; conscientiousness, chi-square(4) = 26.45, p < .001; neuroticism, chi-square(4) = 21.47, p < .001; openness, chi-square(4) = 10.50, p = .033. The global statistic was chi-square(20) = 175.45, p < .001. Bootstrap-calibrated p values, which recompute the loadings in every resample and therefore do not treat them as known, agreed closely: p = .0005 (the resolution floor with B = 2,000) for A, C, E and N, and p = .032 for openness. Openness is the one domain whose rejection is marginal, and it is also the domain with the weakest reliability, so it should be read as suggestive rather than decisive.

Where the misfit sits. Nine of the 25 items had a nuance residual (observed minus loading-predicted age correlation) whose bootstrap 95% CI excluded zero. The largest are readily interpretable. Within agreeableness, "Am indifferent to the feelings of others" (reverse-keyed) has the weakest loading in its domain (lambda = .38) yet the strongest age correlation (r = .152, 95% CI [.115, .191]), a nuance of +.086, 95% CI [.048, .124]; the best indicator of the domain, "Know how to comfort others" (lambda = .76), has a much weaker age correlation than its loading requires (r = .068, nuance = -.065, 95% CI [-.105, -.027]). Within extraversion the split is sharper still: "Find it difficult to approach others" (reverse-keyed) is more strongly age-related than predicted (r = .122, nuance = +.045, 95% CI [.005, .083]) and "Take charge" likewise (r = .102, nuance = +.046, 95% CI [.006, .085]), while "Know how to captivate people" (r = -.007, nuance = -.069, 95% CI [-.108, -.028]) and "Make friends easily" (r = .005, nuance = -.067, 95% CI [-.110, -.028]) are essentially unrelated to age despite loading .60 and .69 on the domain. In conscientiousness, "Continue until everything is perfect" is flat with age (r = .019, nuance = -.070, 95% CI [-.114, -.029]) while the rest of the domain rises. In neuroticism, "Often feel blue" is the outlier (r = -.025, nuance = +.052, 95% CI [.011, .098]). Across all 25 items the root-mean-square item-age correlation was .089 and the root-mean-square nuance residual was .042, so 47% of the typical item-age association, in RMS terms, is not explicable by the domain factor.

One pre-registered diagnostic came back negative and is reported as such: there were no within-domain sign reversals whose confidence interval excluded zero. Items such as "Know how to captivate people" and "Make friends easily" have age correlations indistinguishable from zero rather than significantly opposite in sign, so the misfit takes the form of heterogeneous magnitudes, not of items moving in opposite directions. The rejection therefore rests on the loading-conditioned test rather than on the cruder sign argument.

The nuance profile is reliable, not noise. Across 500 seeded random split halves the correlation between the 25-value nuance profiles computed in the two independent halves averaged r = .788, 95% percentile interval [.664, .887], with a Spearman-Brown-corrected value of .880. Item-specific age information in this instrument is about as reproducible within sample as a well-constructed scale.

Practical magnitude: domain scores discard more than half the age signal. In seeded 10-fold cross-validation with ordinary least squares, out-of-fold R-squared for predicting standardised age was .0369 from the five unit-weighted domain scores, .0285 from five per-domain Thurstone factor scores with weights fitted in the training folds only, and .0840 from the 25 items - corresponding to predicted-observed correlations of r = .193, .170 and .292. The increment from items over domain scores was Delta-R-squared = .047, 95% CI [.028, .066] (percentile bootstrap over participants, B = 4,000), Cohen's f-squared = .051; a paired test on the out-of-fold squared errors gave t(2,400) = 4.63, p < .001. The item model explained 2.27 times as much out-of-sample variance in age as the domain model. Critically, the increment over optimally weighted factor scores was larger still (Delta-R-squared = .055), which rules out the interpretation that items win merely because they are weighted better than a unit-weighted composite: factor scores are the optimal domain-general summary and they performed worst of the three. The result is not an artefact of a lucky fold assignment - across 20 alternative fold seeds the mean out-of-fold R-squared values were .038 (domains), .030 (factor scores) and .085 (items), with the item-minus-domain increment varying by only SD = .0014.

Robustness. Partialling gender out of every item and out of age left the nuance profile essentially unchanged (profile r = .998) and the restriction still rejected in all five domains (gender-adjusted chi-square(4) = 63.57 for A, 25.98 for C, 49.93 for E, 28.82 for N, and 10.00 for O, p = .040), so the effect is not a by-product of the sample's 67% female composition.

Power. No new data were collected, so the appropriate analysis is sensitivity rather than a priori power. With n = 2,401, alpha = .05 two-tailed and 80% power, the smallest detectable correlation is |r| = .057, comfortably below the item-age correlations of interest; 19 of the 25 individual item-age correlations had bootstrap CIs excluding zero. For the 4-df chi-square tests, the critical value is 9.49 and a noncentrality parameter of 11.94 is required for 80% power, which every domain except openness (Q = 10.50) exceeds by a wide margin. The study was therefore well powered for the primary test in four of five domains and only marginally powered in openness.

Verdict. All three pre-specified criteria were met - the restriction was rejected (in five of five domains, not merely one), the nuance profile replicated across split halves with a CI far from zero, and the cross-validated increment was reliably positive - so the hypothesis is supported. The substantive conclusion is that in a 25-item Big Five short form, age differences are not domain-general: roughly half of the item-level age signal, and more than half of the out-of-sample predictive information about age, lives below the domain and is destroyed by the ordinary practice of scoring five domain means. The absolute magnitudes remain small in social-science terms (R-squared = .084 even with all 25 items), so the claim is about what a domain score discards, not about personality being a strong marker of age.

Key numbers

MetricValueInterpretation
Analysed samplen = 2,401 of 2,800 (43 excluded by the 14-70 age filter, 364 for incomplete item responses, 8 overlapping; 399 excluded in total)All exclusions were pre-specified and are reported exactly; the analysed sample is 85.8% of the archive.
Global test of the domain-general restrictionchi-square(20) = 175.45, p < .001; bootstrap-calibrated p = .0005Pooled across the five domains, item-age correlations are not proportional to item loadings - the common-cause model of age differences is rejected.
Per-domain minimum-distance testsA chi-square(4) = 65.06, p < .001; E 51.97, p < .001; C 26.45, p < .001; N 21.47, p < .001; O 10.50, p = .033The restriction fails in every domain. Only openness is marginal, and it is also the least reliable scale (alpha = .61).
Split-half reliability of the nuance profilemean r = .788, 95% percentile interval [.664, .887] over 500 random splits; Spearman-Brown = .880The pattern of item-specific age associations is a reproducible feature of the sample, not sampling noise.
Cross-validated R-squared for predicting agedomain scores .037; Thurstone factor scores .029; 25 items .084 (predicted-observed r = .193, .170 and .292)Out of sample, the raw items carry 2.27 times as much age-related information as the five domain scores computed from those same items.
Incremental explained variance, items over domain scoresDelta-R-squared = .047, 95% CI [.028, .066]; Cohen's f-squared = .051; paired t(2,400) = 4.63, p < .001The increment is reliably positive but small in absolute terms; it is the share discarded, not the total, that is large.
Increment over optimally weighted factor scoresDelta-R-squared = .055 (items .084 vs factor scores .029)Rules out the differential-reliability explanation: the best domain-general summary performed worst, so items do not merely win by being weighted better.
Nuance residuals distinguishable from zero9 of 25 items; RMS nuance .042 against RMS item-age correlation .089 (47%)Roughly half the typical item-age association, in RMS terms, cannot be attributed to the domain factor.
Within-domain sign reversals0 items with an age correlation of opposite sign to the domain whose 95% CI excluded zeroA negative result, reported as such: the misfit is heterogeneity of magnitude (some items flat with age, others not), not opposing directions.
Domain-level age correlationsA .176, C .115, N -.110, E .077, O .068Reproduce the maturity principle at the magnitude published for cross-sectional internet samples, confirming the data behave normally before the critical test.
Gender robustnessnuance profile correlation between gender-adjusted and unadjusted estimates r = .998; restriction still rejected in all five domains (A 63.57, C 25.98, E 49.93, N 28.82, O 10.00, p = .040)The 67% female composition of the sample does not generate the effect.
Fold-seed stabilityAcross 20 alternative CV seeds: mean R-squared .038 (domains), .030 (factor scores), .085 (items); SD of the item-minus-domain increment = .0014The prediction contrast does not depend on a lucky partition.
Sensitivity analysisWith n = 2,401, alpha = .05 two-tailed, 80% power: smallest detectable |r| = .057; chi-square(4) critical value 9.49 with noncentrality 11.94 needed for 80% powerWell powered for four of five domains; openness (Q = 10.50) sits close to the 80%-power boundary.

Caveats

  • The design is cross-sectional. Every quantity here is an age DIFFERENCE, not a within-person change; birth cohort, differential attrition and age-varying self-selection into an online personality study are fully confounded with age and cannot be separated in these data.
  • This is a single self-selected internet sample collected by the SAPA project in Spring 2010, 67% female, with a median age of 26 and very thin coverage above 55. It is not representative of any defined population, and no independent replication sample was analysed.
  • Self-report only. Item-specific variance carries acquiescent and evaluative response bias as well as substantive nuance variance, and nothing in this design separates them; an item-level age effect could in part be an age difference in how people use a six-point scale rather than in what they are like. Informant reports would be needed to distinguish these.
  • The openness result is the weakest link: alpha = .61, the smallest Q (10.50) and a bootstrap p of .032, below the noncentrality of 11.94 that 80% power at 4 df requires. It should be treated as suggestive.
  • The minimum-distance statistic conditions on estimated loadings. The asymptotic chi-square treats them as known, which would be anticonservative; the bootstrap calibration recomputes loadings in every resample and gave materially the same p values, but with B = 2,000 the smallest reportable bootstrap p is .0005, so the four strong rejections are censored at that floor rather than precisely estimated.
  • Complete-case analysis discarded 364 respondents with any missing item. Missingness was scattered across items (9 to 36 per item) and was not modelled; if item non-response is related to both age and item content, the analysed sample is mildly selected.
  • Age was modelled as a linear correlate throughout. Published work finds non-monotonic age trajectories, so a curvilinear specification could change the size, though not the existence, of the heterogeneity; no non-linear specification was run.
  • Absolute predictive accuracy is low: even all 25 items explain only 8.4% of the out-of-sample variance in age. The finding concerns the proportion of age information that domain scoring destroys, and should not be read as personality being a good predictor of age.
  • The one-factor-per-domain measurement model is itself an approximation. Where a five-item domain actually contains two facets, the restriction will fail for that reason - which is a form of the substantive claim (structure below the domain matters), but it means the test cannot distinguish facet-level from item-level specificity in an instrument that measures no facets.

5.Paper writing

Attempt 5Done

Manuscript drafted: “Domain Scores Discard Half the Age Signal in Their Own Items: A Loading-Conditioned Test of the Domain-General Model” (≈ 2800 words).

Reviewer feedback applied: The referee recommends MAJOR REVISION. Rewrite the manuscript to address every required change below, quoted verbatim. Do not resubmit the previous text. (1) "Cut the manuscript to within the venue's 2,000-2,800 word range. Concrete targets: compress the second and third Introduction paragraphs into one; delete the Procedure subsection's restatement and fold its single sentence into Participants; and reduce Table 3 to the nine items whose nuance residual CI excludes zero plus the four largest non-significant deviations, reporting the remaining items' values in the note or stating that the full 25-item table is available with the analysis script." (2) "Add an explicit multiplicity statement to the H1 results: note that with 25 residual CIs at the 95% level roughly 1.25 would be expected to exclude zero by chance and that nine did, and make the same point for the five per-domain tests (the global chi-square(20) = 175.45 already provides the family-wise test - say so)." (3) "Disclose in the Analysis Plan that the split-half procedure re-estimates loadings and item-age correlations within each half but applies the full-sample bootstrap weight matrix V in both, and state that the split halves are therefore not fully independent." (4) "State that the five nuance residuals within a domain carry four independent degrees of freedom, so the 25-value profile correlation is over 20 independent quantities." (5) "Soften the priority claim: replace 'To our knowledge it has not been tested' with a statement that the cited literature reports prediction contrasts rather than a test of the proportionality restriction." (6) "Report bootstrap p values to APA convention as p < .001 and add a table note explaining that .0005 is the smallest attainable value at B = 2,000; and state explicitly that the bootstrap calibration and the weight matrix V come from the same set of resamples." (7) "Revise the Abstract to state that the restriction was rejected in all five domains and that openness was the marginal case (chi-square(4) = 10.50, p = .033), within the 150-word limit." (8) "Move the linear-age limitation from the middle of the limitations paragraph to its own sentence early in the Discussion, and say plainly that heterogeneity in the shape of the age function cannot be distinguished from heterogeneity in linear slope with the analysis that was run." Also: the previous draft ran ~3,690 words; the venue range is 2,000-2,800. Every number must still match the executed RESULT_JSON and every quantitative result must still appear in a numbered Markdown table.

≈ 2800 words · AI-drafted manuscript · not peer-reviewed

Domain Scores Discard Half the Age Signal in Their Own Items: A Loading-Conditioned Test of the Domain-General Model

1. Abstract

Do people differ with age on broad personality domains, or on the specific characteristics domain scores average over? We tested the domain-general (common-cause) model in archival responses from 2,401 Synthetic Aperture Personality Assessment participants who completed a 25-item Big Five inventory. If age relates to an item only through its domain factor, item–age correlations within a domain must be proportional to the items' loadings. That over-identifying restriction was rejected in all five domains — marginally for openness, χ2(4)=10.50\chi^2(4) = 10.50, p = .033 — and globally, χ2(20)=175.45\chi^2(20) = 175.45, p < .001. The residual profile replicated across random split halves (r = .788, 95% CI [.664, .887]), and out of sample the 25 items explained 8.4% of the variance in age against 3.7% for their domain scores, ΔR2=.047\Delta R^2 = .047, 95% CI [.028, .066].

2. Statement of Relevance

Almost every study relating personality to age, health or wellbeing averages a questionnaire's items into five Big Five domain scores and discards the items, a step treated as harmless tidying. Using an openly archived dataset of 2,401 adults who answered 25 standard personality questions, we show it is not. The five domain scores explained less than half as much variance in age as the very same 25 answers did, and the pattern of what each question adds can be recovered from either half of the sample. Researchers who already collect items are discarding usable signal at no benefit, and archives publishing only scale totals have destroyed information nobody can recover. Reporting items alongside domain scores costs nothing and would make personality findings markedly more informative.

3. Introduction

Age differences in the Big Five are among psychology's better-replicated findings (Roberts et al., 2006; Soto et al., 2011), yet both contain an anomaly: facets of one domain move in opposite directions. Mõttus et al. (2017) followed it downward, placing nuances — the specific variance of single items — below facets and showing that this variance behaves like a trait, not error (McCrae & Mõttus, 2019). Mõttus and Rozgonjuk (2021) predicted age at r = .28 from domains, .44 from facets and .65 from the items of a 300-item inventory, and Hang et al. (2021) replicated that ordering (N = 51,524).

That evidence is descriptive: items beating domains is what a purely domain-general world also predicts, since items differ in loading and an optimally weighted composite must beat an unweighted one. The cited work reports prediction contrasts, not a test of the model — which makes a sharper prediction. If age reaches item ii only through the domain factor FdF_d, then with standardised variables

r(itemi, age)=λi βd,r(\text{item}_i,\ \text{age}) = \lambda_i \, \beta_d ,

with λi\lambda_i the loading and βd=corr(Fd,age)\beta_d = \mathrm{corr}(F_d, \text{age}). For a five-item scale that is an over-identifying restriction on 4 df whose rejection cannot be blamed on unequal loadings or reliabilities. That evidence also comes from 240- to 300-item inventories, not the 20–30 item short forms most researchers administer.

We tested three predictions: H1, the restriction is rejected in at least one domain; H2, the residual profile replicates across split halves; H3, the items beat their own domain scores out of sample. Any one of the three failing would leave the domain-general account standing.

Open-science statement. The study was not preregistered; it is a secondary analysis of an archived dataset. Filters, keying, estimators, seeds and refutation criteria were fixed before the estimates were computed (Simmons et al., 2011). The data are public (URL below), one script produced every number, and all effects carry 95% CIs.

4. Method

Participants

Participants were real people; no new data were collected. We analysed the archived bfi dataset from the R package psych (Revelle, Northwestern University), mirrored by Rdatasets at https://vincentarelbundock.github.io/Rdatasets/csv/psych/bfi.csv (retrieved 8 September 2026; SHA-256 68ae71a96c2157b0…): 2,800 observations collected over the web in spring 2010 by the Synthetic Aperture Personality Assessment (SAPA) project, a self-selected volunteer study (Revelle et al., 2010). Two pre-specified exclusions applied — age outside 14–70 and any missing item — leaving n = 2,401. The sample being fixed, we report sensitivity: at α\alpha = .05 two-tailed and 80% power the smallest detectable correlation is ∣r∣|r| = .057, and the 4-df tests need a noncentrality of 11.94 (Table 1).

Materials

Twenty-five International Personality Item Pool items (Goldberg et al., 2006), five per domain, answered from 1 (very inaccurate) to 6 (very accurate); the seven negatively keyed items (A1, C4, C5, E1, E2, O2, O5) were reverse-scored as 7−x7-x. Age is in years and gender 1 = male, 2 = female; education (a 1–5 band) was unused.

Procedure

Self-administered online in spring 2010; no part of it was under our control.

Analysis Plan

Items and age were z-standardised; nothing was removed beyond the two exclusions. Loadings came from iterated principal-axis factoring of each domain's 5 × 5 item correlation matrix (squared multiple correlations as starting communalities), and the 25 × 25 sampling covariance V\mathbf{V} of the item–age correlations from a seeded bootstrap over participants (BB = 2,000) that recomputed loadings in every resample. H1 was tested per domain by minimum distance,

Qd=min⁡β (rd−λdβ)⊤Vd−1(rd−λdβ) ∼ χ2(4),Q_d = \min_{\beta}\ (\mathbf{r}_d - \boldsymbol{\lambda}_d\beta)^{\top}\mathbf{V}_d^{-1}(\mathbf{r}_d - \boldsymbol{\lambda}_d\beta) \ \sim\ \chi^2(4),

plus a global statistic on 20 df. Bootstrap-calibrated p values from recentred resamples keep sampling error in λ\boldsymbol{\lambda} from being ignored, but they use the same resamples that produced V\mathbf{V}. Nuance residuals are ei=ri−λiβ^de_i = r_i - \lambda_i\hat\beta_d; five residuals fitted around one free constant leave four independent residual df per domain, so the 25-value profile carries 20 independent quantities. H2 used 500 seeded split halves, recomputing loadings and correlations within each half but applying the full-sample V\mathbf{V} in both, so the halves are not fully independent. H3 used seeded 10-fold cross-validated ordinary least squares predicting standardised age from five unit-weighted domain scores, five Thurstone factor scores (weights fitted in training folds only), and the 25 items; the increment received a percentile bootstrap CI (BB = 4,000) and was checked across 20 fold seeds. All correlations were also recomputed with gender partialled out.

Transparency and Openness

Every number here came from one executed script run on the archived file; nothing unexecuted is reported.

5. Results

Table 1. Sample, cross-validated prediction of age, profile reliability, and sensitivity.

QuantityValue
Archived / excluded / analysed2,800 / 43 age + 364 incomplete (8 overlap) = 399 / 2,401
Age, analysed sample14–68 years; M = 28.76, SD = 10.76, Mdn = 26 (raw 3–86)
Women / men1,608 (66.97%) / 793
Out-of-fold R2R^2 (r)domains .037 (.193), factor scores .029 (.170), items .084 (.292)
ΔR2\Delta R^2 over domains / over factor scores.047, 95% CI [.028, .066], f2f^2 = .051, ratio 2.27 / .055
20-seed check, mean R2R^2.038 / .030 / .085; SD of increment .0014
Split-half nuance-profile r (500 splits).788, 95% CI [.664, .887]; Spearman–Brown .880
Sensitivity at 80% power$

Note. Data: psych::bfi, SAPA project, spring 2010, retrieved 8 September 2026. Ten-fold cross-validation, ordinary least squares, fixed seed; factor-score weights fitted in training folds only. ΔR2\Delta R^2 CI: percentile bootstrap, B = 4,000; paired test on out-of-fold squared errors t(2,400) = 4.63, p < .001.

H1: The restriction fails in every domain

Domain-score age correlations ran from .176 (agreeableness) to .068 (openness), yet the restriction was rejected in all five domains (Table 2). The family-wise test is the global statistic, χ2(20)=175.45\chi^2(20) = 175.45, p < .001, so the five per-domain tests need no separate multiplicity adjustment. Partialling out gender changed nothing; adjusted and unadjusted residual profiles correlated r = .998.

Table 2. Reliability, age associations, and tests of the domain-general restriction, by domain.

ACENOGlobal
α\alpha.72.74.77.82.61—
r(score, age).176.115.077−.110.068—
β^d\hat\beta_d.173.142.104−.129.090—
χ2\chi^265.0626.4551.9721.4710.50175.45
p< .001< .001< .001< .001.033< .001
χ2\chi^2, gender-adjusted63.5725.9849.9328.8210.00—

Note. A, C, E, N, O = agreeableness, conscientiousness, extraversion, neuroticism, openness; 4 df per domain, 20 global. β^d\hat\beta_d = implied domain-factor–age correlation; gender-adjusted p ≤ .040 throughout. Bootstrap-calibrated p agree (openness .032; elsewhere at the .0005 floor attainable with B = 2,000, hence < .001) but come from the same resamples that produced V\mathbf{V}.

Nine of the 25 nuance residuals had bootstrap 95% CIs excluding zero, against the ≈ 1.25 expected by chance, and the pattern is interpretable (Table 3). Agreeableness's weakest indicator, "Am indifferent to the feelings of others" (λ\lambda = .38), carried its strongest age correlation, while its strongest, "Know how to comfort others" (λ\lambda = .76), fell far below prediction; in extraversion two items rose with age while "Captivate people" and "Make friends easily" stayed flat despite loadings of .60 and .69. The RMS item–age correlation was .089 and the RMS residual .042: 47% of the typical association is not attributable to the domain factor. A pre-specified diagnostic was negative: no item's age correlation reversed its domain's sign with a CI excluding zero, so the misfit is heterogeneity of magnitude, not direction.

Table 3. The nine items whose nuance residual excludes zero (n = 2,401).

Item (abbreviated stem)λ\lambdar(item, age)Nuance e [95% CI]
A1 (R) indifferent to feelings.38.152.086 [.048, .124]
A3 comfort others.76.068−.065 [−.105, −.027]
A4 love children.50.150.063 [.025, .098]
C2 continue till perfect.62.019−.070 [−.114, −.029]
E2 (R) hard to approach others.74.122.045 [.005, .083]
E3 captivate people.60−.007−.069 [−.108, −.028]
E4 make friends easily.69.005−.067 [−.110, −.028]
E5 take charge.54.102.046 [.006, .085]
N4 often feel blue.60−.025.052 [.011, .098]

Note. (R) = reverse-scored; λ\lambda = one-factor principal-axis loading; the model predicts r = λβ^d\lambda\hat\beta_d (Table 2); e = observed − predicted, percentile bootstrap CI, B = 2,000. All 25 were tested; the 16 omitted have CIs containing zero (largest O5 .043, C4 .040, N5 −.035, O4 −.023; the other 12 |e| ≤ .020) and accompany the script. Nineteen of 25 item–age CIs exclude zero (A1's [.115, .191]).

H2: The profile is reliable

Across 500 seeded split halves the residual profiles correlated r = .788, 95% percentile interval [.664, .887], Spearman–Brown .880 (Table 1). Because the profile carries 20 independent df rather than 25 and the halves share the full-sample weight matrix, this is a mild upper bound.

H3: Domain scores discard most of the signal

Out-of-fold R2R^2 was .037 from the five domain scores, .029 from Thurstone factor scores and .084 from the 25 items (Table 1). The increment of items over domain scores was ΔR2=.047\Delta R^2 = .047, 95% CI [.028, .066], f2f^2 = .051, t(2,400) = 4.63, p < .001 — 2.27 times the domain scores' share. The increment over optimally weighted factor scores was larger still: the best domain-general summary came last of the three, ruling out the reading that items win merely by being better weighted.

6. Discussion

All three predictions were met, so the domain-general account of age differences is rejected for this instrument and sample. This is not the familiar observation that items predict better than scales; it is a failure of the model that would have made it innocuous. Conditional on loading, age reaches the items through channels the domain factor does not carry — and optimally weighted factor scores being the worst predictor closes off the differential-reliability explanation.

One qualification belongs up front. Age entered every model linearly, yet the trajectories Roberts et al. (2006) and Soto et al. (2011) document are non-monotonic. The analysis we ran cannot distinguish heterogeneity in the shape of the age function from heterogeneity in linear slope, so the mechanism behind the rejection is less well identified than the rejection itself.

A descriptive regularity (Mõttus & Rozgonjuk, 2021; Hang et al., 2021) thus becomes a rejected model, and the nuance account (Mõttus et al., 2017) reaches a short form where such signal was easy to miss: 56% of the out-of-sample age information in these 25 answers vanishes on averaging.

Limitations. The design is cross-sectional: these are age differences, not change, with cohort and age-varying self-selection confounded. The data are one self-selected internet sample — 67% female, median age 26, thin above 55, no replication — so every claim here is about this class of instrument and sample. Self-report cannot separate nuance variance from acquiescent or evaluative response style (McCrae & Mõttus, 2019). Openness is the weak link (α\alpha = .61, p = .033, under-powered), and complete-case analysis dropped 364 respondents without modelling missingness. One factor per domain is an approximation: a domain holding two facets fails the restriction for that reason, and an instrument with no facet scores cannot separate facet- from item-level specificity, which Olaru et al. (2024) show matters. Absolute accuracy is low (R2R^2 = .084): the claim concerns the share of age information domain scoring destroys, not personality as an age marker.

Constraints on Generality. Following Simons et al. (2017), we expect these results to hold for English-language, self-administered, Likert-format Big Five short forms answered by the kind of self-selected adult internet volunteers SAPA recruited in 2010. We have no evidence for other languages, informant or interview administration, forced-choice formats, clinical or child samples, ages outside 14–68, or inventories that measure facets. Which items carry nuance-level age signal is probably instrument-specific; the general claim — that domain scores are a lossy summary of the age information in their own items — is what we expect to replicate, its descriptive form already having cross-cultural support (Hang et al., 2021).

7. References

Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40(1), 84–96. https://doi.org/10.1016/j.jrp.2005.08.007

Hang, Y., Soto, C., Speyer, L. G., Haring, L., Lee, B., Ostendorf, F., & Mõttus, R. (2021). Age differences in the personality hierarchy: A multi-sample replication study across the life span. Journal of Research in Personality, 93, 104121. https://doi.org/10.1016/j.jrp.2021.104121

McCrae, R. R., & Mõttus, R. (2019). What personality scales measure: A new psychometrics and its implications for theory and assessment. Current Directions in Psychological Science, 28(4), 415–420. https://doi.org/10.1177/0963721419849559

Mõttus, R., Kandler, C., Bleidorn, W., Riemann, R., & McCrae, R. R. (2017). Personality traits below facets: The consensual validity, longitudinal stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 112(3), 474–490. https://doi.org/10.1037/pspp0000100

Mõttus, R., & Rozgonjuk, D. (2021). Development is in the details: Age differences in the Big Five domains, facets, and nuances. Journal of Personality and Social Psychology, 120(4), 1035–1048. https://doi.org/10.1037/pspp0000276

Olaru, G., Jankowsky, K., & Allemand, M. (2024). Structural and inter-individual differentiation in personality traits across the adult lifespan. European Journal of Personality, 38(2), 306–325. https://doi.org/10.1177/08902070231171736

Revelle, W., Wilt, J., & Rosenthal, A. (2010). Individual differences in cognition: New methods for examining the personality–cognition link. In A. Gruszka, G. Matthews, & B. Szymura (Eds.), Handbook of individual differences in cognition (pp. 27–49). Springer. https://doi.org/10.1007/978-1-4419-1210-7_2 [Dataset: bfi, R package psych]

Roberts, B. W., Walton, K. E., & Viechtbauer, W. (2006). Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies. Psychological Bulletin, 132(1), 1–25. https://doi.org/10.1037/0033-2909.132.1.1

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632

Simons, D. J., Shoda, Y., & Lindsay, D. S. (2017). Constraints on Generality (COG): A proposed addition to all empirical papers. Perspectives on Psychological Science, 12(6), 1123–1128. https://doi.org/10.1177/1745691617708630

Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2011). Age differences in personality traits from 10 to 65: Big Five domains and facets in a large cross-sectional sample. Journal of Personality and Social Psychology, 100(2), 330–348. https://doi.org/10.1037/a0021717

6.Review

Attempt 2Done

Automated review: Minor revision (novelty 4, rigor 4, clarity 4, reproducibility 5).

Minor revision
Novelty 4/5Rigor 4/5Clarity 4/5Reproducibility 5/5

This revision addresses every point raised previously and does so substantively rather than cosmetically. The manuscript is now within the venue's 2,000-2,800 word range with no executed result dropped; multiplicity is handled by naming the global chi-square(20) = 175.45 as the family-wise test and by setting the nine significant item-level intervals against the approximately 1.25 expected by chance; the four independent degrees of freedom per domain and the resulting 20-quantity profile are stated; the split halves are disclosed as sharing the full-sample weight matrix; bootstrap p values are reported as < .001 with the B = 2,000 resolution floor explained and their dependence on the same resamples as V made explicit; the priority claim is softened to a statement about what the cited literature does and does not test; the abstract now reports rejection in all five domains with openness marginal, chi-square(4) = 10.50, p = .033; and the linear-age limitation has been moved to its own early paragraph in the Discussion with a plain statement that curvature and slope heterogeneity cannot be separated by the analysis run. The science is unchanged and remains sound: the restriction is rejected globally, chi-square(20) = 175.45, p < .001, and in every domain; the nuance profile replicates across split halves, r = .788, 95% CI [.664, .887]; and the out-of-sample increment of items over their own domain scores is reliable if modest, Delta-R-squared = .047, 95% CI [.028, .066], with optimally weighted factor scores predicting worst of the three model classes. Every number, filter, seed and provenance detail matches the executed script and the archived file, and the respondents are correctly described as real people in a 2010 SAPA collection. What remains is presentational: an omnibus Table 1 that mixes sample descriptives with cross-validation results, two abbreviated item stems quoted as if verbatim, a dangling antecedent and an unexplained ratio introduced by the compression, and an assertion of low power in the Discussion that has lost its supporting statistic. None of these affect a single reported quantity. Minor revision.

Strengths

  • The contribution is genuinely methodological rather than descriptive. Prior work (Mottus & Rozgonjuk, 2021; Hang et al., 2021) shows that items out-predict domain scores, which a purely domain-general world also predicts. Deriving r_i = lambda_i * beta_d and testing it as an over-identifying restriction with 4 df per five-item domain turns that regularity into a falsifiable model test, and the manuscript now states plainly that the cited literature reports prediction contrasts rather than a test of the model, so no priority claim is overreached.
  • The Thurstone factor-score comparison remains the strongest single piece of evidence and is correctly interpreted: optimally weighted domain-general scores predicted age worst of the three model classes (out-of-fold R-squared .029 against .037 for unit-weighted composites and .084 for the items), which closes the differential-reliability objection empirically as well as analytically.
  • Every reported quantity matches the executed script and the data provenance. The dataset (psych::bfi, SAPA project, spring 2010, retrieved 8 September 2026, SHA-256 prefix given), 2,800 loaded observations, the two exclusions (43 by the 14-70 age filter, 364 for incomplete items, 8 overlapping, 399 total), n = 2,401, the six-point item metric, the seven reverse-keyed items, B = 2,000 and 4,000, 500 split halves, 10 folds and 20 fold seeds, and every chi-square, correlation, CI and R-squared correspond to the record. Participants are correctly described as real respondents in an archival dataset and the data are nowhere called simulated.
  • The four analytic disclosures that were missing before are now present and correctly placed: the family-wise chi-square(20) = 175.45 is named as the multiplicity control for the five per-domain tests, the nine of 25 item-level intervals are set against the approximately 1.25 expected by chance, the five nuance residuals are stated to carry four independent degrees of freedom so that the profile correlation is over 20 quantities, and the split halves are disclosed as sharing the full-sample weight matrix V and therefore as not fully independent.
  • Bootstrap p values are now reported to APA convention as < .001, with a table note explaining that .0005 is the resolution floor at B = 2,000, and the note states that the calibration comes from the same resamples that produced V. Reporting is otherwise complete for a short report: exact ns, a sensitivity analysis appropriate to a fixed archival sample, 95% CIs on every effect, fixed seeds, and a candid statement that declines to claim preregistration.
  • Negative results are reported rather than buried. The pre-specified sign-reversal diagnostic returned zero and is stated as such, openness is flagged repeatedly as the marginal domain, and the abstract now says that the restriction was rejected in all five domains with openness the marginal case. The Constraints on Generality statement is specific and genuinely constraining.
  • The manuscript is now within the venue's length range (about 2,800 words) without losing any executed result, and the Introduction reads cleanly for a broad audience.

Weaknesses

  • Table 1 is an omnibus that mixes three unrelated things - sample composition and exclusions, the cross-validated prediction results for H3, and the sensitivity power analysis. Sample descriptives that belong in Method are now printed in Results ahead of H1, and a reader looking for the H3 results has to scroll back past the participant counts to find them. This looks like a casualty of the length cut rather than a design choice.
  • Two abbreviated item stems are placed inside quotation marks in the H1 paragraph ('Captivate people', 'Make friends easily'). The IPIP stems are 'Know how to captivate people' and 'Make friends easily'; quoting a truncation as if verbatim is a small accuracy problem in a paper whose argument turns on what individual items say.
  • 'That evidence also comes from 240- to 300-item inventories' has lost its antecedent in the compression. The sentence now follows a passage about the degrees of freedom of the restriction, so 'that evidence' has no referent in its own paragraph.
  • '2.27 times the domain scores' share' is loose. The intended comparison (out-of-fold R-squared .084 against .037) is in Table 1 but the sentence does not say what the ratio is a ratio of.
  • The openness limitation in the Discussion now gives only alpha = .61, p = .033 and the word 'under-powered'. The quantitative form of the point - chi-square(4) = 10.50 against the noncentrality of 11.94 required for 80% power - has been dropped from the text, and 'under-powered' without it is an assertion.
  • 'RMS' is used twice in the Results without root-mean-square being spelled out first.
  • The substantive limits are unchanged and remain severe in aggregate: a single cross-sectional self-selected internet sample, 67% female, median age 26, thin above 55, complete-case, self-report only, with no replication sample. The manuscript now says explicitly that every claim is about this class of instrument and sample, which is the right response, but the study cannot do more than that.
  • Age is modelled linearly throughout while the two age papers the manuscript cites document non-monotonic trajectories. The Discussion now states early and plainly that heterogeneity in the shape of the age function cannot be distinguished from heterogeneity in linear slope with the analysis that was run - the correct disclosure, but it does leave the mechanism behind the rejection unidentified.
  • One factor per domain is an approximation the data cannot test in a five-item scale, so part of the rejected misfit may be facet structure rather than item-level nuance. This is acknowledged with the appropriate citation, but it caps how strongly the 'nuance' interpretation can be pressed.

Required changes

  • Split Table 1. Put sample composition and exclusions in a Method table (or a Participants paragraph), and keep the cross-validated prediction, split-half reliability and sensitivity results in a Results table placed with H2 and H3.
  • Quote item stems verbatim or drop the quotation marks: 'Captivate people' should read 'Know how to captivate people', and the abbreviated forms in Table 3 should be labelled in the note as abbreviations of the published stems.
  • Repair the antecedent in 'That evidence also comes from 240- to 300-item inventories' - name the prediction-contrast literature explicitly, since the preceding sentences are about the restriction rather than about evidence.
  • State the comparison behind the 2.27 ratio in words (out-of-fold R-squared .084 for the items against .037 for the domain scores) rather than 'the domain scores' share'.
  • Restore the quantitative form of the openness limitation in the Discussion: chi-square(4) = 10.50 against the noncentrality of 11.94 required for 80% power, so that 'under-powered' is supported rather than asserted.
  • Spell out root-mean-square at first use before abbreviating it to RMS.

Record

§Event timeline

  1. Run createdQuestion: “Are age differences in personality domain-general or nuance-specific? Testing whether Big Five domain scores capture the age-related information carried by their own items” · output language: English · venue: Psychology · Psychological Science · models: agent:claude-opus-5
  2. Stage started · Literature reviewAttempt 1 · agent:claude-opus-5
  3. awaiting_agent · Literature reviewsearch response for literature_review: literature_review-search-1.response.json
  4. Resume requested
  5. Stage started · Literature reviewAttempt 2 · agent:claude-opus-5
  6. Search complete · Literature review18 sources · 11,125 characters of notes
  7. awaiting_agent · Literature reviewcomplete_json response for literature_review: literature_review-complete_json-1.response.json
  8. Resume requested
  9. Stage started · Literature reviewAttempt 3 · agent:claude-opus-5
  10. Search complete · Literature review18 sources · 11,125 characters of notes
  11. Stage completed · Literature review
  12. Stage started · IdeationAttempt 1 · agent:claude-opus-5
  13. awaiting_agent · Ideationcomplete_json response for ideation: ideation-complete_json-1.response.json
  14. Resume requested
  15. Stage started · IdeationAttempt 2 · agent:claude-opus-5
  16. Awaiting approval · Ideation
  17. Approved · IdeationIdea 1 selected
  18. Stage started · ExperimentationAttempt 1 · agent:claude-opus-5
  19. Dataset provisioned · Experimentationbfi_personality_items · 2,800 observations · 29 variables · sha256 68ae71a96c21
  20. awaiting_agent · Experimentationcomplete response for experimentation: experimentation-complete-1.response.txt
  21. Resume requested
  22. Stage started · ExperimentationAttempt 2 · agent:claude-opus-5
  23. Dataset provisioned · Experimentationbfi_personality_items · 2,800 observations · 29 variables · sha256 68ae71a96c21
  24. Experiment round · ExperimentationRound 1: exit 0; result parsed (9.137s)
  25. Stage completed · Experimentation
  26. Stage started · AnalysisAttempt 1 · agent:claude-opus-5
  27. awaiting_agent · Analysiscomplete_json response for analysis: analysis-complete_json-1.response.json
  28. Resume requested
  29. Stage started · AnalysisAttempt 2 · agent:claude-opus-5
  30. Stage completed · Analysis
  31. Stage started · Paper writingAttempt 1 · agent:claude-opus-5
  32. awaiting_agent · Paper writingcomplete response for paper_writing: paper_writing-complete-1.response.txt
  33. Resume requested
  34. Stage started · Paper writingAttempt 2 · agent:claude-opus-5
  35. Stage completed · Paper writing
  36. Stage started · ReviewAttempt 1 · agent:claude-opus-5
  37. awaiting_agent · Reviewcomplete_json response for review: review-complete_json-1.response.json
  38. Resume requested
  39. Stage started · ReviewAttempt 2 · agent:claude-opus-5
  40. Awaiting approval · Review
  41. Sent back for revision · ReviewRe-running from Paper writing: “The referee recommends MAJOR REVISION. Required changes, quoted verbatim:\n(1) "Cut the manuscript to within the venue\'s 2,000-2,800 word range. Concrete targets: compress the second and third Introduc”
  42. Stage started · Paper writingAttempt 3 · agent:claude-opus-5
  43. Stage completed · Paper writing
  44. Stage started · ReviewAttempt 1 · agent:claude-opus-5
  45. Awaiting approval · Review
  46. Sent back for revision · ReviewRe-running from Paper writing: “The referee recommends MAJOR REVISION. Rewrite the manuscript to address every required change below, quoted verbatim. Do not resubmit the previous text.\n(1) "Cut the manuscript to within the venue\'s ”
  47. Stage started · Paper writingAttempt 4 · agent:claude-opus-5
  48. awaiting_agent · Paper writingcomplete response for paper_writing: paper_writing-complete-1.response.txt
  49. Resume requested
  50. Stage started · Paper writingAttempt 5 · agent:claude-opus-5
  51. Stage completed · Paper writing
  52. Stage started · ReviewAttempt 1 · agent:claude-opus-5
  53. awaiting_agent · Reviewcomplete_json response for review: review-complete_json-1.response.json
  54. Resume requested
  55. Stage started · ReviewAttempt 2 · agent:claude-opus-5
  56. Awaiting approval · Review
  57. Approved · Review
  58. Run completereport=/private/tmp/claude-501/-Users-haodingresearch-Desktop-auto-research/579004ae-5819-417d-918c-9d9409997567/scratchpad/studies/psychology-en/runs/44d2d5b89d84/REPORT.md