Auto-Research PipelineA durable, human-gated pipeline for computational research

← All examples

Research rund0124d4c52b0· 24 October 2026

Are age differences in personality domain-general or nuance-specific? Testing whether Big Five domain scores capture the age-related information carried by their own items

CompleteArchived runPsychology · Modelled on Psychological Science

Paper

Four Big Five Domains Do Not Carry All of Their Items' Age Differences: A Loading-Based Test in 25 IPIP Items

Abstract

Do age differences in personality belong to Big Five domains or to the characteristics their items describe? If age acted on items only through their domain, item–age associations would be proportional to factor loadings. We tested this restriction with multiple-indicators multiple-causes models in 2,209 adults (18–65 years) who answered 25 International Personality Item Pool items. It was rejected for agreeableness, conscientiousness, extraversion and neuroticism, χ²(4) = 17.32 to 53.59, bootstrap ps ≤ .004, also under acceptably fitting measurement models; only for neuroticism did a loading-free test also reject, so the other three rest on the estimated loadings. Openness was undetermined (p = .073; .014 to .456 in other specifications). Departures replicated across halves (median r = .78). Out of sample, the 25 items predicted age better than the five domain scores, R² = .067, 95% CI [.043, .089], versus .026 [.010, .040], but beat factor scores within only two domains.

Keywordspersonality developmentage differencesBig Fivepersonality nuancesmeasurement invarianceMIMIC modelscross-validation

§Ask about this paper

Grounded in this run

Questions are answered from this run's record — the paper, the executed analysis, the data provenance and the referee report. Nothing else is consulted.

This run is finished, so the recorded paper cannot be rewritten. Questions are still answered from its record.

Pipeline

1.Literature review

Attempt 3Done

Literature review complete (live:agent:claude-opus-5-5): 71 key papers, 74 grounded sources, 6 open gaps.

Summary

Age differences in personality are conventionally described at the level of the Big Five domains: meta-analyses and large samples show maturation (rising conscientiousness, agreeableness and emotional stability across adulthood; Roberts et al., 2006; Bleidorn et al., 2022; Graham et al., 2020; Soto et al., 2011), and the two leading theories (Five-Factor Theory's intrinsic maturation and the social-investment perspective) are stated for domains, implying that a domain develops as a unit. A second line of work contradicts this picture from below. Facets within a domain follow different age trends (Roberts et al., 2006; Soto et al., 2011), items of the same domain fail intercept invariance across age groups (Mõttus et al., 2015), item-specific variance (personality nuances) is reliable, consensually valid and heritable (McCrae, 2015; Mõttus et al., 2017, 2019), and out-of-sample age prediction improves sharply from domains (r = .28) to facets (.44) to items (.65) in a 300-item inventory (Mõttus & Rozgonjuk, 2021), a pattern replicated across samples, ages and instruments (Hang et al., 2021) and consistent with age-related differences concentrated in item residuals (Olaru et al., 2024). Measurement-invariance studies, by contrast, usually conclude that domain comparisons across age are defensible (Nye et al., 2016; Brandt et al., 2020; Zimprich et al., 2012). The two literatures have not been put on a common footing. The decisive methodological point is that prediction contrasts are not a test of the domain-general account: unit-weighted domain scores are a constrained factor model (McNeish & Wolf, 2020), so items can out-predict them even when age acts only through the domain. A domain-general (structural) account makes a sharper, falsifiable prediction - item-age associations must be proportional to the items' factor loadings, i.e., no direct age effects in a MIMIC model (VanderWeele & Vansteelandt, 2022; Jöreskog & Goldberger, 1975; Muthén, 1989) - and this restriction has not, to our search, been tested for age. Nor has the nuance result been established for short inventories with five items per domain, the format most studies use. Competing explanations for item-specific age differences that a test must address are facet structure, nonlinear age curves, acquiescence and education, social desirability and cohort. Psychological Science reporting expects estimation with confidence intervals for every effect, exact p values, disclosed exclusions and analytic decisions, and a sample-size (sensitivity) justification.

Key papers (71)

  1. Development is in the details: Age differences in the Big Five domains, facets, and nuances. Mõttus, R., & Rozgonjuk, D. - Journal of Personality and Social Psychology, 120(4), 1035-1048, 2021.
    Direct precursor: in N = 24,000 (ages 18-50, 300 items), out-of-sample age prediction r = .28 from domains, .44 facets, .65 items; residualising items for facets did not reduce accuracy - age information mostly nuance-specific.
  2. Age differences in the personality hierarchy: A multi-sample replication study across the life span. Hang, Y., Soto, C., Speyer, L. G., Haring, L., Lee, B., Ostendorf, F., & Mõttus, R. - Journal of Research in Personality, 93, 104121, 2021.
    Multi-sample replication (N = 51,524; different ages, cultures, instruments): lower hierarchy levels, especially nuances, hold substantially more age-sensitive information than domains.
  3. Within-trait heterogeneity in age group differences in personality domains and facets: Implications for the development and coherence of personality traits. Mõttus, R., Realo, A., Allik, J., Esko, T., Metspalu, A., & Johnson, W. - PLOS ONE, 10(3), e0119667, 2015.
    N = 2,711, NEO PI-3, 12 age groups 18-91: no facet or domain met strong (intercept) invariance; 46% of items had significant residual age correlations; argues domains are not aetiologically coherent units of development.
  4. Personality traits below facets: The consensual validity, longitudinal stability, heritability, and utility of personality nuances. Mõttus, R., Kandler, C., Bleidorn, W., Riemann, R., & McCrae, R. R. - Journal of Personality and Social Psychology, 112(3), 474-490, 2017.
    Defines nuances as item residuals; shows cross-rater agreement, stability and heritability of item-specific variance; few robust age associations in that twin sample.
  5. Personality characteristics below facets: A replication and meta-analysis of cross-rater agreement, rank-order stability, heritability, and utility of personality nuances. Mõttus, R., Sinick, J., Terracciano, A., Hřebíčková, M., Kandler, C., Ando, J., Mortensen, E. L., Colodro-Conde, L., & Jang, K. L. - Journal of Personality and Social Psychology, 117(4), e35-e50, 2019.
    Replication in 6,287 people from 6 countries; item-residual associations with age, gender and BMI replicated across samples - replicability as the validity criterion for nuances.
  6. A more nuanced view of reliability: Specificity in the trait hierarchy. McCrae, R. R. - Personality and Social Psychology Review, 19(2), 97-112, 2015.
    Theoretical basis for nuances: item-specific variance is reliable, observable and probably heritable; traits as intersection versus union of constituents.
  7. What personality scales measure: A new psychometrics and its implications for theory and assessment. McCrae, R. R., & Mõttus, R. - Current Directions in Psychological Science, 28(4), 415-420, 2019.
    States the view that scales measure many specific characteristics as well as a common factor.
  8. Structural and inter-individual differentiation in personality traits across the adult lifespan. Olaru, G., Jankowsky, K., & Allemand, M. - European Journal of Personality, 38(2), 306-325, 2024.
    LSEM in three samples aged 16-90: domain loadings stable across age, item-residual variances differ; most age differences affect only the item level.
  9. Age differences in personality traits and social desirability: A multi-rater multi-sample study. Ausmees, L., Kandler, C., Realo, A., Allik, J., Borkenau, P., Hřebíčková, M., & Mõttus, R. - Journal of Research in Personality, 99, 104245, 2022.
    Age differences are smaller in informant than self reports, especially for desirable traits - a non-substantive source of item-level age differences.
  10. A statistical test to reject the structural interpretation of a latent factor model. VanderWeele, T. J., & Vansteelandt, S. - Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(5), 2032-2054, 2022.
    Core logic of the present test: under a structural univariate factor model, associations of indicators with any external variable are proportional to their loadings; loading-based and reliability-independent tests; continuous Z left as future work.
  11. Estimation of a model with multiple indicators and multiple causes of a single latent variable. Jöreskog, K. G., & Goldberger, A. S. - Journal of the American Statistical Association, 70(351a), 631-639, 1975.
    The MIMIC model used to express the domain-general restriction.
  12. Latent variable modeling in heterogeneous populations. Muthén, B. O. - Psychometrika, 54(4), 557-585, 1989.
    MIMIC models with direct covariate effects on indicators for heterogeneity/DIF.
  13. Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis. Woods, C. M. - Multivariate Behavioral Research, 44(1), 1-27, 2009.
    Methodological basis for testing direct (DIF) effects of a covariate in MIMIC models.
  14. Age differences in the symptoms of depression: A latent trait analysis. Gallo, J. J., Anthony, J. C., & Muthén, B. O. - Journal of Gerontology, 49(6), P251-P264, 1994.
    Classic MIMIC application with age as the covariate and direct effects on specific symptoms.
  15. A more general model for testing measurement invariance and differential item functioning. Bauer, D. J. - Psychological Methods, 22(3), 507-526, 2017.
    Moderated nonlinear factor analysis for continuous moderators such as age.
  16. Thinking twice about sum scores. McNeish, D., & Wolf, M. G. - Behavior Research Methods, 52(6), 2287-2305, 2020.
    Sum scores are a constrained factor model; explains why items can out-predict unit-weighted domain scores without any item-specific information.
  17. Thinking thrice about sum scores, and then some more about measurement and analysis. Widaman, K. F., & Revelle, W. - Behavior Research Methods, 55(2), 788-806, 2023.
    Counterpoint defending sum scores in many applications.
  18. Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies. Roberts, B. W., Walton, K. E., & Viechtbauer, W. - Psychological Bulletin, 132(1), 1-25, 2006.
    Canonical domain-level maturation evidence (92 samples); extraversion had to be split into social dominance and social vitality because they diverge.
  19. Personality stability and change: A meta-analysis of longitudinal studies. Bleidorn, W., Schwaba, T., Zheng, A., Hopwood, C. J., Sosa, S. S., Roberts, B. W., & Briley, D. A. - Psychological Bulletin, 148(7-8), 588-619, 2022.
    Updated meta-analysis: maturation-direction mean-level change; facet-level measures less stable than domain measures.
  20. Trajectories of Big Five personality traits: A coordinated analysis of 16 longitudinal samples. Graham, E. K., Weston, S. J., Gerstorf, D., Yoneda, T. B., Booth, T., Beam, C. R., et al. - European Journal of Personality, 34(3), 301-321, 2020.
    Coordinated growth models in 16 samples (> 60,000): declines in C, E, O and nonlinear late-life increases in N; heterogeneous trajectories.
  21. Age differences in personality traits from 10 to 65: Big Five domains and facets in a large cross-sectional sample. Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. - Journal of Personality and Social Psychology, 100(2), 330-348, 2011.
    N = 1,267,218 internet sample: adult trends toward maturity, but facets within a domain often show distinct age trends.
  22. Development of personality in early and middle adulthood: Set like plaster or persistent change?. Srivastava, S., John, O. P., Gosling, S. D., & Potter, J. - Journal of Personality and Social Psychology, 84(5), 1041-1053, 2003.
    N = 132,515 internet sample aged 21-60; C and A rise through adulthood; varied patterns of change.
  23. Personality maturation around the world: A cross-cultural examination of social-investment theory. Bleidorn, W., Klimstra, T. A., Denissen, J. J. A., Rentfrow, P. J., Potter, J., & Gosling, S. D. - Psychological Science, 24(12), 2530-2540, 2013.
    Target-venue paper: maturation in young adults across 62 nations (N = 884,328), timing linked to social roles.
  24. Personality across the life span. Costa, P. T., Jr., McCrae, R. R., & Löckenhoff, C. E. - Annual Review of Psychology, 70, 423-448, 2019.
    Review of life-span personality development.
  25. Personality trait change in adulthood. Roberts, B. W., & Mroczek, D. - Current Directions in Psychological Science, 17(1), 31-35, 2008.
    Short review of adult trait change.
  26. Nature over nurture: Temperament, personality, and life span development. McCrae, R. R., Costa, P. T., Jr., Ostendorf, F., Angleitner, A., Hřebíčková, M., Avia, M. D., et al. - Journal of Personality and Social Psychology, 78(1), 173-186, 2000.
    Five-Factor Theory's intrinsic-maturation account, a domain-level theory of development.
  27. Evaluating Five Factor Theory and social investment perspectives on personality trait development. Roberts, B. W., Wood, D., & Smith, J. L. - Journal of Research in Personality, 39(1), 166-184, 2005.
    Social-investment account of development, also stated at the trait level.
  28. Stability and change of personality across the life course: The impact of age and major life events on mean-level and rank-order stability of the Big Five. Specht, J., Egloff, B., & Schmukle, S. C. - Journal of Personality and Social Psychology, 101(4), 862-882, 2011.
    National panel evidence on age and life events.
  29. Personality development across the life span: Longitudinal analyses with a national sample from Germany. Lucas, R. E., & Donnellan, M. B. - Journal of Personality and Social Psychology, 101(4), 847-861, 2011.
    Longitudinal national-panel age trends.
  30. Age differences in the Big Five across the life span: Evidence from two national samples. Donnellan, M. B., & Lucas, R. E. - Psychology and Aging, 23(3), 558-566, 2008.
    Cross-sectional national-sample age differences.
  31. Between facets and domains: 10 aspects of the Big Five. DeYoung, C. G., Quilty, L. C., & Peterson, J. B. - Journal of Personality and Social Psychology, 93(5), 880-896, 2007.
    Hierarchy below domains (aspects).
  32. The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Soto, C. J., & John, O. P. - Journal of Personality and Social Psychology, 113(1), 117-143, 2017.
    Facet structure within Big Five domains.
  33. Measurement invariance of Big-Five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and la dolce vita effects. Marsh, H. W., Nagengast, B., & Morin, A. J. S. - Developmental Psychology, 49(6), 1194-1218, 2013.
    Item-level CFA of a short BFI often fails; hybrid MIMIC/multigroup invariance over ages 15-99; strong nonlinear age effects.
  34. Personality trait differences between young and middle-aged adults: Measurement artifacts or actual trends?. Nye, C. D., Allemand, M., Gosling, S. D., Potter, J., & Roberts, B. W. - Journal of Personality, 84(4), 473-492, 2016.
    Invariance between ages 20 and 50 (n = 15,726 each): non-equivalence negligible.
  35. Personality across the lifespan: Exploring measurement invariance of a short Big Five Inventory from ages 11 to 84. Brandt, N. D., Becker, M., Tetzner, J., Brunner, M., Kuhl, P., & Maaz, K. - European Journal of Psychological Assessment, 36(1), 162-173, 2020.
    Short-inventory invariance across age: comparable structure, loadings and most intercepts.
  36. Factorial structure and age-related psychometrics of the MIDUS personality adjective items across the life span. Zimprich, D., Allemand, M., & Lachman, M. E. - Psychological Assessment, 24(1), 173-186, 2012.
    Strict loading invariance across 10 age groups with age differences in factor means.
  37. Successful explanations start with accurate descriptions: Questionnaire items as personality markers for more accurate predictions. Seeboth, A., & Mõttus, R. - European Journal of Personality, 32(3), 186-201, 2018.
    Items out-predict Big Five domains for 40 outcomes (about 30% more variance).
  38. The finer details? The predictability of life outcomes from Big Five domains, facets, and nuances. Stewart, R. D., Mõttus, R., Seeboth, A., Soto, C. J., & Johnson, W. - Journal of Personality, 90(2), 167-182, 2022.
    Domains 16.6%, facets 18.0%, items 20.9% of outcome variance on average.
  39. Towards more rigorous personality trait-outcome research. Mõttus, R. - European Journal of Personality, 30(4), 292-303, 2016.
    Argues for item-level checks of whether trait associations are domain-wide.
  40. Descriptive, predictive and explanatory personality research: Different goals, different approaches, but a shared need to move beyond the Big Few traits. Mõttus, R., Wood, D., Condon, D. M., Back, M. D., Baumert, A., Costantini, G., et al. - European Journal of Personality, 34(6), 1175-1201, 2020.
    Framework for going beyond domains.
  41. Exploring the persome: The power of the item in understanding personality structure. Revelle, W., Dworak, E. M., & Condon, D. M. - Personality and Individual Differences, 169, 109905, 2021.
    Argues for the item as a unit of personality structure (SAPA authors).
  42. When are item nuances useful for prediction in organizations? Comparing the validity of item-level, scale-level, and ensemble machine learning models. Tang, C., Hickman, L., Song, Q. C., & Alexander, L. - Journal of Applied Psychology (advance online publication), 2025.
    Conditions under which item-level models beat scale models (few nuance items, large nuance effects, high consistency, large samples).
  43. The developmental psychometrics of Big Five self-reports: Acquiescence, factor structure, coherence, and differentiation from ages 10 to 20. Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. - Journal of Personality and Social Psychology, 94(4), 718-737, 2008.
    Acquiescence varies with age and distorts Big Five self-report structure.
  44. Acquiescence in personality questionnaires: Relevance, domain specificity, and stability. Danner, D., Aichholzer, J., & Rammstedt, B. - Journal of Research in Personality, 57, 119-130, 2015.
    Acquiescence as a stable, partly domain-specific response style.
  45. The measurement equivalence of Big-Five factor markers for persons with different levels of education. Rammstedt, B., Goldberg, L. R., & Borg, I. - Journal of Research in Personality, 44(1), 53-61, 2010.
    Education-related differences in structure linked to acquiescence.
  46. Cognitive ability, acquiescence, and the structure of personality in a sample of older adults. Lechner, C. M., & Rammstedt, B. - Psychological Assessment, 27(4), 1301-1311, 2015.
    Lower cognitive ability goes with stronger acquiescence in older adults; controlling acquiescence restores the structure.
  47. A general model for the study of developmental problems. Schaie, K. W. - Psychological Bulletin, 64(2), 92-107, 1965.
    Age, cohort and period are confounded in cross-sectional designs.
  48. Thinking clearly about correlations and causation: Graphical causal models for observational data. Rohrer, J. M. - Advances in Methods and Practices in Psychological Science, 1(1), 27-42, 2018.
    Causal language and assumptions for observational associations such as cross-sectional age differences.
  49. Choosing prediction over explanation in psychology: Lessons from machine learning. Yarkoni, T., & Westfall, J. - Perspectives on Psychological Science, 12(6), 1100-1122, 2017.
    Cross-validated prediction and overfitting in psychology.
  50. Cross-validation: A method every psychologist should know. de Rooij, M., & Weeda, W. - Advances in Methods and Practices in Psychological Science, 3(2), 248-263, 2020.
    K-fold cross-validation practice for psychologists.
  51. When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. - Psychological Methods, 17(3), 354-373, 2012.
    Justifies treating six-point items as continuous with robust ML.
  52. Bootstrapping goodness-of-fit measures in structural equation models. Bollen, K. A., & Stine, R. A. - Sociological Methods & Research, 21(2), 205-229, 1992.
    Model-based bootstrap that calibrates chi-square tests under non-normality.
  53. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Hu, L., & Bentler, P. M. - Structural Equation Modeling, 6(1), 1-55, 1999.
    Fit-index conventions (CFI, RMSEA, SRMR).
  54. Comparative fit indexes in structural models. Bentler, P. M. - Psychological Bulletin, 107(2), 238-246, 1990.
    CFI.
  55. Structural model evaluation and modification: An interval estimation approach. Steiger, J. H. - Multivariate Behavioral Research, 25(2), 173-180, 1990.
    RMSEA.
  56. Regression modeling strategies (2nd ed.). Harrell, F. E., Jr. - Springer, 2015.
    Restricted cubic splines for flexible age effects.
  57. A simple sequentially rejective multiple test procedure. Holm, S. - Scandinavian Journal of Statistics, 6(2), 65-70, 1979.
    Family-wise error control across the five domain tests.
  58. Reliability from alpha to omega: A tutorial. Revelle, W., & Condon, D. M. - Psychological Assessment, 31(12), 1395-1411, 2019.
    Reliability coefficients for short scales.
  59. The theoretical status of latent variables. Borsboom, D., Mellenbergh, G. J., & van Heerden, J. - Psychological Review, 110(2), 203-219, 2003.
    Realist interpretation of latent variables that the structural test probes.
  60. Dimensions of normal personality as networks in search of equilibrium: You can't like parties if you don't like people. Cramer, A. O. J., van der Sluis, S., Noordhof, A., Wichers, M., Geschwind, N., Aggen, S. H., Kendler, K. S., & Borsboom, D. - European Journal of Personality, 26(4), 414-431, 2012.
    Network alternative in which items are distinct components.
  61. Worse than measurement error: Consequences of inappropriate latent variable measurement models. Rhemtulla, M., van Bork, R., & Borsboom, D. - Psychological Methods, 25(1), 30-45, 2020.
    Bias when a reflective model is imposed on a different structure.
  62. Measurement schmeasurement: Questionable measurement practices and how to avoid them. Flake, J. K., & Fried, E. I. - Advances in Methods and Practices in Psychological Science, 3(4), 456-465, 2020.
    Measurement transparency standards.
  63. Hidden invalidity among 15 commonly used measures in social and personality psychology. Hussey, I., & Hughes, S. - Advances in Methods and Practices in Psychological Science, 3(2), 166-184, 2020.
    Structural validity of widely used measures is rarely tested.
  64. The new statistics: Why and how. Cumming, G. - Psychological Science, 25(1), 7-29, 2014.
    Venue reporting standard: estimation and confidence intervals.
  65. Business not as usual. Eich, E. - Psychological Science, 25(1), 3-6, 2014.
    Venue disclosure standards (exclusions, analytic decisions).
  66. Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. - American Psychologist, 73(1), 3-25, 2018.
    JARS quantitative reporting standards.
  67. The international personality item pool and the future of public-domain personality measures. Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. - Journal of Research in Personality, 40(1), 84-96, 2006.
    Describes the IPIP from which the 25 items come.
  68. Individual differences in cognition: New methods for examining the personality-cognition link. Revelle, W., Wilt, J., & Rosenthal, A. - In A. Gruszka, G. Matthews, & B. Szymura (Eds.), Handbook of individual differences in cognition (pp. 27-49). Springer, 2010.
    Data-collection source of the bfi data (SAPA, spring 2010).
  69. A SAPA project update: On the structure of phrased self-report personality items. Condon, D. M., Roney, E., & Revelle, W. - Journal of Open Psychology Data, 5(1), 3, 2017.
    Describes the SAPA project and its item pool.
  70. bfi: 25 Personality items representing 5 factors. Documentation of the psych package data set (Revelle, W.), Rdatasets mirror.
    Verbatim item stems, response scale, coding of gender, education and age; collected spring 2010; not John's BFI.
  71. psych: Procedures for psychological, psychometric, and personality research. Revelle, W. - R package, Northwestern University (CRAN).
    Distributes the bfi data set.

Methods in use

  • Domain-level mean trajectories from multilevel growth models (longitudinal) or polynomial/nonlinear age models (cross-sectional)
  • Measurement invariance across age: multiple-group CFA, ESEM, MIMIC direct effects, local SEM and moderated nonlinear factor analysis
  • Item residualisation on domain/facet scores followed by residual-age correlations at a stringent alpha (e.g., p < .0002)
  • Cross-validated age prediction from domains, facets and items (penalised regression; train/test partitions), reported as r or R-squared between predicted and observed age
  • Replication of item-specific effects across independent samples, raters or split halves as the validity criterion for nuances
  • Structural-interpretation (proportionality) test: item-external-variable associations proportional to factor loadings (VanderWeele & Vansteelandt, 2022)
  • Robust ML treating six-point items as continuous, Bollen-Stine bootstrap for non-normal chi-square tests, CFI/RMSEA/SRMR fit indices
  • Restricted cubic splines for flexible age effects; Holm correction across domain-level tests
  • Psychological Science reporting: effect sizes with 95% CIs, exact p values, disclosed exclusions, sensitivity power for the available n

Open gaps

  • No formal latent-variable test of whether domain scores carry all the age information in their own items: prediction contrasts confound item-specific information with the equal weighting of sum scores and with overfitting
  • Nuance-specific age evidence comes from 240-300-item inventories; whether a short inventory with five items per domain also loses most item-level age information is untested
  • Residualising items on fallible domain scores leaves domain variance in the residuals and biases toward finding item-specific age effects
  • Invariance studies conclude domain comparisons across age are defensible while nuance studies conclude most age information is item-specific; the two have not been reconciled with one test
  • Separating substantive nuances from facet structure, nonlinear item-specific age curves, acquiescence/education and social desirability
  • Replicability of item-specific age deviations within and across samples, and cohort confounding of cross-sectional age differences
Grounded search sources (74) · mode: live:agent:claude-opus-5-5

2.Ideation

Attempt 2Done

3 candidate ideas generated; selected: “Do Big Five domains carry their items' age information? A structural (MIMIC) test of domain-general age differences in a 25-item inventory”.

  1. Idea 1Selected

    Do Big Five domains carry their items' age information? A structural (MIMIC) test of domain-general age differences in a 25-item inventory

    Hypothesis. Age differences in personality are not domain-general. A domain-general account implies that age is related to a domain's items only through the domain's common factor, so each item's association with age must be proportional to its factor loading (the structural-interpretation restriction of VanderWeele & Vansteelandt, 2022; no direct age effects in a MIMIC model). H1 (primary): this restriction is rejected (Holm-adjusted bootstrap p < .05) in at least four of the five Big Five domains. H2: out of sample, a domain's five items predict age better than the domain's optimally weighted factor score, and the five unit-weighted domain scores recover less than half of the age variance the 25 items predict. H3: item-specific age deviations replicate across random halves of the sample. H1 is refuted if the restriction is retained in two or more domains.

    Dataset. Big Five Inventory: 25 personality items, SAPA project bfi_personality_items

    Test plan, novelty and risk

    Test plan. Data: bfi_personality_items (SAPA, 25 IPIP items). Sample: participants aged 18-65 with all 25 item responses; report the flow from 2,800 (exclusions for age < 18, age > 65, any missing item) and the analysed n (about 2,200). Items reverse-keyed (7 - x) for A1, C4, C5, E1, E2, O2, O5; domain score = mean of its five keyed items; six-point items treated as continuous (normal-theory ML) with resampling-based inference. Descriptives: item-age and domain-age Pearson r with 95% CIs (Fisher z); alpha and omega; fit of each one-factor measurement model (chi-square(5), CFI, RMSEA, SRMR). H1, per domain: model H0 (domain-general) x = nu + lambda*eta + e, eta = gamma*age + zeta (age acts only through the factor) versus model HA with free item-specific age slopes and the same one-factor residual structure; ML likelihood-ratio test, chi-square(4), with a Bollen-Stine-type bootstrap p value (residuals transformed to the H0-implied covariance, data regenerated from the fitted H0 mean structure, seeded) and Holm correction across the five domains. Effect size: the non-proportional share of the item-age correlation profile, NPS = 1 - cos^2(r, l), where r are item-age correlations and l standardized loadings (0 under H0), with a nonparametric bootstrap 95% CI and its distribution under the fitted H0 model; per-item deviations from the domain-implied correlation with bootstrap CIs. Sensitivity: the single-item age deviation (correlation metric) detectable with 80% power at the Holm-adjusted alpha for the analysed n, from the noncentral chi-square. Robustness (all executed, not merely planned): (a) age as a restricted cubic spline (4 knots), testing proportionality of whole age curves (chi-square(12)); (b) a loading-free rank-one test that all five items share one age curve (canonical-correlation test, chi-square(8)); (c) adjustment for gender, education (dummies with a missing category) and a leave-domain-out acquiescence index (mean raw response to balanced positively/negatively keyed item pairs from the other domains), each with free direct effects; (d) the item pair with the largest residual correlation freed as a correlated uniqueness, to absorb facet-like dependence. H2: repeated 10-fold cross-validation (10 repeats, seeded) predicting age by OLS from (i) a domain's unit-weighted score, (ii) its Bartlett factor score with weights estimated in the training folds, (iii) its five items; and overall from five domain scores, five factor scores and all 25 items; out-of-sample R-squared with participant bootstrap 95% CIs for each model and for differences. Factor score vs items isolates item-specific information; unit score vs factor score isolates the cost of equal weighting. H3: 100 random half-splits; item deviations estimated in each half; cross-half Pearson correlation across the 25 items (median, 2.5th-97.5th percentiles) against an age-permutation null; Spearman-Brown full-sample value. Refutation: H1 refuted if the restriction is retained (Holm-adjusted p >= .05) in two or more domains; H2 refuted if the items-minus-factor-score R-squared differences are null (CIs include 0) in most domains; H3 refuted if the median cross-half correlation does not exceed the 95th percentile of the permutation null.

    Novelty. Prior nuance studies (Mõttus & Rozgonjuk, 2021; Hang et al., 2021) used 240-300-item inventories and prediction contrasts, which confound item-specific information with the equal weighting of sum scores (McNeish & Wolf, 2020) and with overfitting; residualising items on fallible domain scores is biased toward nuance findings. This study applies the falsifiable proportionality restriction implied by a structural, domain-general account to age, with a continuous-age MIMIC adaptation of the VanderWeele-Vansteelandt test, calibrated by bootstrap, in the short five-items-per-domain format most studies use, and separates weighting from nuance in out-of-sample prediction.

    Expected outcome. Rejection of the domain-general restriction in most domains, with item-specific deviations that replicate across halves and survive flexible age, covariate/acquiescence adjustment and a correlated residual; items out-predicting factor scores out of sample, with domain scores recovering well under half of the items' age information. Openness, whose age associations are weakest, is the most likely domain to retain the restriction.

    Main risk. Cross-sectional, self-selected internet sample (age is confounded with cohort; few participants over 55); a five-item scale cannot separate facet-level from item-level (nuance) specificity; one-factor misfit could distort the loadings that the restriction uses; non-substantive item-level age differences (acquiescence, social desirability, age-related item interpretation) cannot be fully excluded with self-reports alone.

  2. Idea 2

    Is item-level age information a response style? Acquiescence and keying as the source of item-specific age differences

    Hypothesis. If item-specific age differences in the 25 IPIP items are an artefact of age-graded acquiescence, then within domains that mix positively and negatively keyed items, items' deviations from their domain-implied age association will differ systematically by keying direction, and partialling a balanced-pair acquiescence index will remove most of the item-specific age variance. H: keying direction explains less than 25% of the variance in item-level age deviations, and partialling acquiescence reduces the items-over-domain predictive gain for age by less than half.

    Dataset. Big Five Inventory: 25 personality items, SAPA project bfi_personality_items

    Test plan, novelty and risk

    Test plan. Data: bfi_personality_items; ages 18-65 with complete items. Acquiescence = mean raw response to balanced positively/negatively keyed item pairs (A1/A2, C1-C2/C4-C5, E1-E2/E3-E4, O1-O3/O2-O5). Estimate item-age correlations and their deviation from the domain-implied value (one-factor loadings), regress deviations on keying direction (with domain fixed effects; bootstrap CI for R-squared), and compare cross-validated item-minus-domain R-squared for age before and after partialling acquiescence (participant bootstrap CI). Refuted if keying explains at least 25% of deviation variance or the predictive gain falls by half or more.

    Novelty. Acquiescence is known to vary with age and education (Soto et al., 2008; Rammstedt et al., 2010; Lechner & Rammstedt, 2015) but has not been tested as the source of nuance-level age differences, which Mõttus & Rozgonjuk (2021) checked only for age-graded item content.

    Expected outcome. Keying explains a small share of deviations; the item-level advantage largely survives partialling acquiescence.

    Main risk. Only seven reverse-keyed items and none in neuroticism; the balanced-pair index is an imperfect acquiescence measure.

  3. Idea 3

    Age differentiation of Big Five measurement: do loadings and item residual variances change with age?

    Hypothesis. Consistent with Olaru et al. (2024), the structure of the five domains is stable across adulthood but item residual variances change with age: H: in each domain, age moderates item residual variances (heteroskedasticity of item residuals with age, joint test p < .05) while standardized loadings vary by less than .10 between ages 20 and 50.

    Dataset. Big Five Inventory: 25 personality items, SAPA project bfi_personality_items

    Test plan, novelty and risk

    Test plan. Data: bfi_personality_items; ages 18-65 with complete items. Local (kernel-weighted) one-factor ML models at focal ages 20, 30, 40, 50 (Gaussian kernel, bandwidth from the age distribution); loadings and residual variances per focal age with bootstrap CIs; permutation test of age moderation (permuting age, 199 permutations). Refuted if loadings change by more than .10 or residual variances show no age moderation.

    Novelty. Applies local SEM-style differentiation tests to a short inventory and links residual-variance change to the nuance debate.

    Expected outcome. Stable loadings; modest age moderation of some item residual variances.

    Main risk. Sparse data above age 50 make focal-age estimates noisy; kernel bandwidth choice affects results.

Ranking rationale

Idea 0 answers the stated question directly with a falsifiable, theory-derived restriction rather than a prediction contrast, separates weighting from nuance, and builds in replication and robustness; ideas 1 and 2 address one alternative explanation each (response style; age-moderated measurement) and are narrower.

3.Experimentation

Attempt 5Done

Experiment succeeded in 1 round (50.336s).

Reviewer feedback applied: Independent referee (Claude Fable 5.1): major_revision. Required changes: (1) [analysis] Report fit indices (chi-square, CFI, RMSEA, SRMR) for the alternative-model measurement structure after the correlated uniqueness is freed, and for conscientiousness and neuroticism re-run the restriction test under a measurement model that fits acceptably (e.g., free residual covariances in order of standardized residual size until RMSEA <= .08, or a two-factor structure if identified), with its bootstrap p value; if no adequately fitting model exists, say so and temper the abstract and title for those two domains. (2) [analysis] Provide multiplicity-adjusted intervals for the 25 item deviations (a simultaneous bootstrap band or Holm-adjusted percentile intervals from the existing 999 resamples) and state how many items survive; confine the item-level statements in the Statement of Relevance and Discussion to items that survive, or label them explicitly as unadjusted and exploratory. (3) [analysis] Add a missing-data sensitivity analysis for the five primary tests using pairwise-available moments or full-information ML on all 2,543 age-eligible participants, and report whether the chi-square statistics and Holm decisions change. (4) [text] State the pre-specified decision rules for H2 and H3 as written before analysis (items beat the factor score in most domains; domain scores recover under half of the items' R-squared; median cross-half r above the permutation 95th percentile) and report H2 against them: the per-domain criterion was met in 2 of 5 domains and the share interval includes .5, so only the aggregate contrast supports H2. Replace 'partly supported' accordingly in Results and Discussion. (5) [text] Make the abstract and Discussion say that the rejections for agreeableness, conscientiousness and extraversion are conditional on the one-factor measurement model and that the loading-free test rejected only for neuroticism; the Robustness section says this but the abstract does not. (6) [text] Reword the theoretical claim: maturation and social-investment theories do not themselves imply proportional item differences; state that the implication follows only when they are read as claims about the domain as a single developing unit (Equation 1), and acknowledge that facet-level accounts (Roberts et al., 2006) are compatible with the rejections. (7) [text] Fix the inconsistencies: describe the conscientiousness fit the same way throughout (poor by the cited cutoffs); give the openness secondary-test range as .036 to .208 or restrict the sentence to the three loading-based tests; replace 'showed no age association' with 'showed no detectable age association (95% CI including zero and excluding the implied value)'. (8) [text] Standardize precision to APA conventions: test statistics to two decimals everywhere (45.15, not 45.152), one rounding rule for every value taken from the output (the extraversion score-age CI should read [.026, .109] under the rule used elsewhere), and keep four-decimal bootstrap p values only in Table 2 with the existing note. (9) [text] In Transparency and Openness give the location (repository or supplement) of the archived script, its full output and the earlier version referred to, or remove the sentence about changes after the first run; report software versions (Python, NumPy, SciPy, pandas). (10) [text] Cite and briefly contrast the retrieved continuous-covariate DIF literature (Bauer, 2017; Muthen, 1989; Gallo et al., 1994) where the MIMIC direct-effects alternative is introduced; cite Widaman and Revelle (2023) where factor scores are found to predict worse than unit scores; add Zimprich et al. (2012) to the invariance evidence and Condon et al. (2017) for the SAPA design and the larger public releases that would permit an external replication. (11) [text] Qualify the 'short inventory most studies use' framing in the Introduction and Constraints on Generality: the 25 items are a demonstration selection of IPIP items, not a validated short Big Five inventory, and the sample is a 2010 demonstration subset of SAPA. (12) [text] Reduce display items to the venue's allowance: move Table 3, Table 6 and the adjusted-deviation column of Table 4 to supplemental material, keeping the key robustness p values in the text, and confirm the main text against the current word limit.
Succeeded1 round · exit 0 · 50.336s

Data provenance

Datasetbfi_personality_items

Big Five Inventory: 25 personality items, SAPA project

Source
R package 'psych' (William Revelle, Northwestern University), mirrored by the Rdatasets archive; data collected by the SAPA personality project
Citation
Items from Goldberg, L.R. (1999) A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In: Mervielde, I. et al. (eds) Personality Psychology in Europe, Vol. 7. Tilburg: Tilburg University Press, 7-28. Data collected in Spring 2010 by the SAPA project: Revelle, W., Wilt, J. and Rosenthal, A. (2010) Individual differences in cognition: new methods for examining the personality-cognition link. In: Gruszka, A., Matthews, G. and Szymura, B. (eds) Handbook of Individual Differences in Cognition. New York: Springer, ch. 2, 27-49. DOI: 10.1007/978-1-4419-1210-7_2. Distributed as bfi in Revelle, W. psych: Procedures for Psychological, Psychometric, and Personality Research. Northwestern University, Evanston, Illinois.
Licence
Redistributed in the R package 'psych' under GPL (>= 2) via the Rdatasets archive. No dataset-specific licence is published; the IPIP items are described as public domain but IPIP issues no formal licence text, and the SAPA project no longer publishes data terms.
File
https://vincentarelbundock.github.io/Rdatasets/csv/psych/bfi.csv
Observations
2,800
Variables (29)
subject_id, A1, A2, A3, A4, A5, C1, C2, C3, C4, C5, E1, E2, E3, E4, E5, N1, N2, N3, N4, N5, O1, O2, O3, O4, O5, gender, education, age
Units
Items A1-A5, C1-C5, E1-E5, N1-N5 and O1-O5 are six-point Likert responses from 1 (very inaccurate) to 6 (very accurate). Several items are negatively keyed and must be reversed before scoring: A1, C4, C5, E1, E2, O2 and O5. gender is 1 male, 2 female. education is a 1-5 ordinal band and age is in years. subject_id is the original SAPA respondent identifier.
Retrieved
24 Oct 2026, 03:04
Checksum
sha256 68ae71a96c2157b0…

Real, publicly available data — downloaded and checksum-verified before the analysis ran, never simulated.

Result (RESULT_JSON)

{
  "flow": {
    "n_total": 2800,
    "excluded_age_under_18": 248,
    "excluded_age_over_65": 9,
    "excluded_incomplete_items_in_age_range": 334,
    "n_analysed": 2209
  },
  "sample": {
    "age_mean": 29.7424,
    "age_sd": 10.3932,
    "age_median": 27,
    "n_age_18_29": 1323,
    "n_age_30_44": 633,
    "n_age_45_65": 253,
    "pct_female": 67.0439,
    "n_education_missing": 31,
    "pct_edu_some_college": 48.0308,
    "pct_edu_college_or_more": 31.8244,
    "mean_age_incomplete_excluded": 31.1976
  },
  "spline_knots_years": [
    18,
    23,
    31,
    51
  ],
  "domains": {
    "A": {
      "primary": {
        "chi2": 45.152,
        "df": 4,
        "p_asymptotic": 3.696822893098304e-9,
        "p_bootstrap": 0.0005
      },
      "null_chi2_p95": 9.2851,
      "nps": 0.2326,
      "nps_ci": [
        0.126,
        0.4398
      ],
      "nps_null_mean": 0.0323,
      "nps_null_p95": 0.0855,
      "domain_score_age_r": 0.1556,
      "domain_score_age_r_ci": [
        0.1146,
        0.196
      ],
      "latent_age_corr": 0.1442,
      "latent_age_corr_ci": [
        0.094,
        0.1915
      ],
      "mde_direct_r_median": 0.0772,
      "mde_direct_r_range": [
        0.076,
        0.0841
      ],
      "spline": {
        "chi2": 51.8294,
        "df": 12,
        "p_asymptotic": 6.646347757661525e-7,
        "p_bootstrap": 0.002
      },
      "adjusted": {
        "chi2": 39.1968,
        "df": 4,
        "p_asymptotic": 6.343943052320152e-8,
        "p_bootstrap": 0.002
      },
      "cu": {
        "chi2": 48.891,
        "df": 4,
        "p_asymptotic": 6.15266395276945e-10,
        "p_bootstrap": 0.002
      },
      "cu_pair": [
        "A1",
        "A2"
      ],
      "cu_resid_corr": 0.0967,
      "rank1": {
        "chi2": 10.9817,
        "df": 8,
        "p_asymptotic": 0.20273816808673795,
        "p_bootstrap": 0.21
      },
      "fit_chi2_df5": 75.4393,
      "fit_cfi": 0.9657,
      "fit_rmsea": 0.0799,
      "fit_srmr": 0.0331,
      "alpha": 0.7086,
      "omega": 0.712,
      "p_holm": 0.0025
    },
    "C": {
      "primary": {
        "chi2": 23.815,
        "df": 4,
        "p_asymptotic": 0.00008699322810840304,
        "p_bootstrap": 0.0005
      },
      "null_chi2_p95": 10.1197,
      "nps": 0.259,
      "nps_ci": [
        0.1035,
        0.6531
      ],
      "nps_null_mean": 0.0663,
      "nps_null_p95": 0.1884,
      "domain_score_age_r": 0.0866,
      "domain_score_age_r_ci": [
        0.045,
        0.1278
      ],
      "latent_age_corr": 0.1063,
      "latent_age_corr_ci": [
        0.0546,
        0.1526
      ],
      "mde_direct_r_median": 0.0785,
      "mde_direct_r_range": [
        0.0771,
        0.0798
      ],
      "spline": {
        "chi2": 30.4476,
        "df": 12,
        "p_asymptotic": 0.0023900273790964863,
        "p_bootstrap": 0.014
      },
      "adjusted": {
        "chi2": 15.9685,
        "df": 4,
        "p_asymptotic": 0.003061760546502222,
        "p_bootstrap": 0.008
      },
      "cu": {
        "chi2": 22.0885,
        "df": 4,
        "p_asymptotic": 0.0001924542095608422,
        "p_bootstrap": 0.004
      },
      "cu_pair": [
        "C1",
        "C2"
      ],
      "cu_resid_corr": 0.0928,
      "rank1": {
        "chi2": 9.4782,
        "df": 8,
        "p_asymptotic": 0.30357103449618145,
        "p_bootstrap": 0.34
      },
      "fit_chi2_df5": 127.4341,
      "fit_cfi": 0.9417,
      "fit_rmsea": 0.1053,
      "fit_srmr": 0.0407,
      "alpha": 0.733,
      "omega": 0.736,
      "p_holm": 0.0025
    },
    "E": {
      "primary": {
        "chi2": 53.5947,
        "df": 4,
        "p_asymptotic": 6.398326460794592e-11,
        "p_bootstrap": 0.0005
      },
      "null_chi2_p95": 9.5634,
      "nps": 0.5669,
      "nps_ci": [
        0.282,
        0.9218
      ],
      "nps_null_mean": 0.1119,
      "nps_null_p95": 0.3387,
      "domain_score_age_r": 0.0671,
      "domain_score_age_r_ci": [
        0.0255,
        0.1085
      ],
      "latent_age_corr": 0.0763,
      "latent_age_corr_ci": [
        0.0295,
        0.1269
      ],
      "mde_direct_r_median": 0.0764,
      "mde_direct_r_range": [
        0.0724,
        0.0785
      ],
      "spline": {
        "chi2": 60.7109,
        "df": 12,
        "p_asymptotic": 1.674451417766053e-8,
        "p_bootstrap": 0.002
      },
      "adjusted": {
        "chi2": 42.8367,
        "df": 4,
        "p_asymptotic": 1.1187627055389044e-8,
        "p_bootstrap": 0.002
      },
      "cu": {
        "chi2": 57.5007,
        "df": 4,
        "p_asymptotic": 9.713531185504935e-12,
        "p_bootstrap": 0.002
      },
      "cu_pair": [
        "E3",
        "E5"
      ],
      "cu_resid_corr": 0.0838,
      "rank1": {
        "chi2": 6.9453,
        "df": 8,
        "p_asymptotic": 0.5425428877852573,
        "p_bootstrap": 0.56
      },
      "fit_chi2_df5": 63.624,
      "fit_cfi": 0.9769,
      "fit_rmsea": 0.0729,
      "fit_srmr": 0.0278,
      "alpha": 0.7669,
      "omega": 0.773,
      "p_holm": 0.0025
    },
    "N": {
      "primary": {
        "chi2": 17.3248,
        "df": 4,
        "p_asymptotic": 0.0016712930202802071,
        "p_bootstrap": 0.0035
      },
      "null_chi2_p95": 9.673,
      "nps": 0.1594,
      "nps_ci": [
        0.0553,
        0.453
      ],
      "nps_null_mean": 0.0432,
      "nps_null_p95": 0.1237,
      "domain_score_age_r": -0.0978,
      "domain_score_age_r_ci": [
        -0.1389,
        -0.0563
      ],
      "latent_age_corr": -0.104,
      "latent_age_corr_ci": [
        -0.1498,
        -0.0559
      ],
      "mde_direct_r_median": 0.0672,
      "mde_direct_r_range": [
        0.0622,
        0.0782
      ],
      "spline": {
        "chi2": 38.1843,
        "df": 12,
        "p_asymptotic": 0.0001432612858580464,
        "p_bootstrap": 0.002
      },
      "adjusted": {
        "chi2": 16.3534,
        "df": 4,
        "p_asymptotic": 0.002579796033803233,
        "p_bootstrap": 0.006
      },
      "cu": {
        "chi2": 20.6722,
        "df": 4,
        "p_asymptotic": 0.0003677481149173922,
        "p_bootstrap": 0.002
      },
      "cu_pair": [
        "N4",
        "N5"
      ],
      "cu_resid_corr": 0.1249,
      "rank1": {
        "chi2": 22.6719,
        "df": 8,
        "p_asymptotic": 0.003812095087094797,
        "p_bootstrap": 0.004
      },
      "fit_chi2_df5": 314.675,
      "fit_cfi": 0.9214,
      "fit_rmsea": 0.1675,
      "fit_srmr": 0.058,
      "alpha": 0.8162,
      "omega": 0.8146,
      "p_holm": 0.007
    },
    "O": {
      "primary": {
        "chi2": 8.3703,
        "df": 4,
        "p_asymptotic": 0.07891789772167628,
        "p_bootstrap": 0.0725
      },
      "null_chi2_p95": 9.2478,
      "nps": 0.2801,
      "nps_ci": [
        0.0915,
        0.8375
      ],
      "nps_null_mean": 0.1841,
      "nps_null_p95": 0.5189,
      "domain_score_age_r": 0.0567,
      "domain_score_age_r_ci": [
        0.015,
        0.0982
      ],
      "latent_age_corr": 0.0741,
      "latent_age_corr_ci": [
        0.0247,
        0.1266
      ],
      "mde_direct_r_median": 0.0837,
      "mde_direct_r_range": [
        0.0834,
        0.086
      ],
      "spline": {
        "chi2": 15.7591,
        "df": 12,
        "p_asymptotic": 0.2025189184787541,
        "p_bootstrap": 0.17
      },
      "adjusted": {
        "chi2": 9.8861,
        "df": 4,
        "p_asymptotic": 0.04239063292196648,
        "p_bootstrap": 0.036
      },
      "cu": {
        "chi2": 9.7392,
        "df": 4,
        "p_asymptotic": 0.04505750169011947,
        "p_bootstrap": 0.046
      },
      "cu_pair": [
        "O2",
        "O5"
      ],
      "cu_resid_corr": 0.0752,
      "rank1": {
        "chi2": 9.597,
        "df": 8,
        "p_asymptotic": 0.2944576695833588,
        "p_bootstrap": 0.208
      },
      "fit_chi2_df5": 67.6236,
      "fit_cfi": 0.9469,
      "fit_rmsea": 0.0753,
      "fit_srmr": 0.0333,
      "alpha": 0.6182,
      "omega": 0.6279,
      "p_holm": 0.0725
    }
  },
  "items": {
    "A1": {
      "r": 0.1416,
      "r_ci": [
        0.1005,
        0.1822
      ],
      "loading_std": 0.361,
      "implied_r": 0.0575,
      "dev": 0.0841,
      "dev_ci": [
        0.0486,
        0.1193
      ],
      "dev_adj": 0.0295,
      "dev_adj_ci": [
        -0.0071,
        0.0653
      ]
    },
    "A2": {
      "r": 0.1068,
      "r_ci": [
        0.0654,
        0.1479
      ],
      "loading_std": 0.6555,
      "implied_r": 0.1049,
      "dev": 0.0019,
      "dev_ci": [
        -0.0263,
        0.0293
      ],
      "dev_adj": -0.0024,
      "dev_adj_ci": [
        -0.0295,
        0.0251
      ]
    },
    "A3": {
      "r": 0.0537,
      "r_ci": [
        0.0121,
        0.0952
      ],
      "loading_std": 0.7676,
      "implied_r": 0.1234,
      "dev": -0.0697,
      "dev_ci": [
        -0.0916,
        -0.0443
      ],
      "dev_adj": -0.0662,
      "dev_adj_ci": [
        -0.0906,
        -0.0396
      ]
    },
    "A4": {
      "r": 0.1216,
      "r_ci": [
        0.0803,
        0.1625
      ],
      "loading_std": 0.4873,
      "implied_r": 0.0779,
      "dev": 0.0437,
      "dev_ci": [
        0.0108,
        0.078
      ],
      "dev_adj": 0.0803,
      "dev_adj_ci": [
        0.0481,
        0.1148
      ]
    },
    "A5": {
      "r": 0.1017,
      "r_ci": [
        0.0603,
        0.1428
      ],
      "loading_std": 0.625,
      "implied_r": 0.1001,
      "dev": 0.0016,
      "dev_ci": [
        -0.028,
        0.0299
      ],
      "dev_adj": 0.0063,
      "dev_adj_ci": [
        -0.0222,
        0.0357
      ]
    },
    "C1": {
      "r": 0.0723,
      "r_ci": [
        0.0307,
        0.1136
      ],
      "loading_std": 0.5449,
      "implied_r": 0.0556,
      "dev": 0.0167,
      "dev_ci": [
        -0.0171,
        0.0512
      ],
      "dev_adj": 0.0139,
      "dev_adj_ci": [
        -0.0202,
        0.047
      ]
    },
    "C2": {
      "r": -0.0011,
      "r_ci": [
        -0.0428,
        0.0406
      ],
      "loading_std": 0.6079,
      "implied_r": 0.0622,
      "dev": -0.0632,
      "dev_ci": [
        -0.0964,
        -0.0312
      ],
      "dev_adj": -0.0543,
      "dev_adj_ci": [
        -0.0888,
        -0.0215
      ]
    },
    "C3": {
      "r": 0.0524,
      "r_ci": [
        0.0107,
        0.0939
      ],
      "loading_std": 0.5574,
      "implied_r": 0.0569,
      "dev": -0.0046,
      "dev_ci": [
        -0.0342,
        0.0234
      ],
      "dev_adj": -0.012,
      "dev_adj_ci": [
        -0.0426,
        0.0183
      ]
    },
    "C4": {
      "r": 0.1172,
      "r_ci": [
        0.0759,
        0.1581
      ],
      "loading_std": 0.6868,
      "implied_r": 0.0698,
      "dev": 0.0474,
      "dev_ci": [
        0.0224,
        0.075
      ],
      "dev_adj": 0.0297,
      "dev_adj_ci": [
        0.0039,
        0.0556
      ]
    },
    "C5": {
      "r": 0.0598,
      "r_ci": [
        0.0182,
        0.1013
      ],
      "loading_std": 0.5932,
      "implied_r": 0.0606,
      "dev": -0.0007,
      "dev_ci": [
        -0.0311,
        0.0286
      ],
      "dev_adj": 0.0186,
      "dev_adj_ci": [
        -0.0108,
        0.0483
      ]
    },
    "E1": {
      "r": 0.0531,
      "r_ci": [
        0.0114,
        0.0946
      ],
      "loading_std": 0.5936,
      "implied_r": 0.0434,
      "dev": 0.0097,
      "dev_ci": [
        -0.0179,
        0.0388
      ],
      "dev_adj": -0.0077,
      "dev_adj_ci": [
        -0.0356,
        0.0234
      ]
    },
    "E2": {
      "r": 0.1088,
      "r_ci": [
        0.0674,
        0.1498
      ],
      "loading_std": 0.7374,
      "implied_r": 0.0536,
      "dev": 0.0552,
      "dev_ci": [
        0.0311,
        0.0828
      ],
      "dev_adj": 0.0484,
      "dev_adj_ci": [
        0.0234,
        0.0741
      ]
    },
    "E3": {
      "r": -0.0208,
      "r_ci": [
        -0.0624,
        0.021
      ],
      "loading_std": 0.6003,
      "implied_r": 0.0439,
      "dev": -0.0647,
      "dev_ci": [
        -0.0939,
        -0.0335
      ],
      "dev_adj": -0.0656,
      "dev_adj_ci": [
        -0.0952,
        -0.0339
      ]
    },
    "E4": {
      "r": -0.0056,
      "r_ci": [
        -0.0473,
        0.0362
      ],
      "loading_std": 0.6937,
      "implied_r": 0.0508,
      "dev": -0.0563,
      "dev_ci": [
        -0.0844,
        -0.0262
      ],
      "dev_adj": -0.0359,
      "dev_adj_ci": [
        -0.0648,
        -0.0064
      ]
    },
    "E5": {
      "r": 0.0986,
      "r_ci": [
        0.0571,
        0.1397
      ],
      "loading_std": 0.5312,
      "implied_r": 0.0387,
      "dev": 0.0599,
      "dev_ci": [
        0.0245,
        0.0924
      ],
      "dev_adj": 0.0613,
      "dev_adj_ci": [
        0.025,
        0.0951
      ]
    },
    "N1": {
      "r": -0.0741,
      "r_ci": [
        -0.1154,
        -0.0325
      ],
      "loading_std": 0.825,
      "implied_r": -0.088,
      "dev": 0.014,
      "dev_ci": [
        -0.0103,
        0.0347
      ],
      "dev_adj": 0.0237,
      "dev_adj_ci": [
        0.0004,
        0.0455
      ]
    },
    "N2": {
      "r": -0.0815,
      "r_ci": [
        -0.1228,
        -0.0399
      ],
      "loading_std": 0.8033,
      "implied_r": -0.0857,
      "dev": 0.0042,
      "dev_ci": [
        -0.0218,
        0.0309
      ],
      "dev_adj": 0.0043,
      "dev_adj_ci": [
        -0.0213,
        0.0303
      ]
    },
    "N3": {
      "r": -0.1008,
      "r_ci": [
        -0.1419,
        -0.0594
      ],
      "loading_std": 0.7202,
      "implied_r": -0.0767,
      "dev": -0.0241,
      "dev_ci": [
        -0.0473,
        -0.0004
      ],
      "dev_adj": -0.0295,
      "dev_adj_ci": [
        -0.054,
        -0.0047
      ]
    },
    "N4": {
      "r": -0.0132,
      "r_ci": [
        -0.0549,
        0.0285
      ],
      "loading_std": 0.5563,
      "implied_r": -0.0595,
      "dev": 0.0463,
      "dev_ci": [
        0.0148,
        0.0771
      ],
      "dev_adj": 0.0335,
      "dev_adj_ci": [
        0.0011,
        0.0663
      ]
    },
    "N5": {
      "r": -0.1001,
      "r_ci": [
        -0.1412,
        -0.0587
      ],
      "loading_std": 0.4959,
      "implied_r": -0.0528,
      "dev": -0.0473,
      "dev_ci": [
        -0.0827,
        -0.0136
      ],
      "dev_adj": -0.0449,
      "dev_adj_ci": [
        -0.0828,
        -0.0095
      ]
    },
    "O1": {
      "r": 0.0408,
      "r_ci": [
        -0.0009,
        0.0824
      ],
      "loading_std": 0.541,
      "implied_r": 0.0399,
      "dev": 0.0009,
      "dev_ci": [
        -0.0329,
        0.0315
      ],
      "dev_adj": 0.0295,
      "dev_adj_ci": [
        -0.0061,
        0.0606
      ]
    },
    "O2": {
      "r": 0.0345,
      "r_ci": [
        -0.0072,
        0.0761
      ],
      "loading_std": 0.4954,
      "implied_r": 0.0366,
      "dev": -0.0021,
      "dev_ci": [
        -0.0329,
        0.0304
      ],
      "dev_adj": -0.038,
      "dev_adj_ci": [
        -0.0682,
        -0.0055
      ]
    },
    "O3": {
      "r": 0.0264,
      "r_ci": [
        -0.0153,
        0.068
      ],
      "loading_std": 0.6325,
      "implied_r": 0.0467,
      "dev": -0.0203,
      "dev_ci": [
        -0.047,
        0.0073
      ],
      "dev_adj": -0.0094,
      "dev_adj_ci": [
        -0.0377,
        0.0183
      ]
    },
    "O4": {
      "r": -0.0071,
      "r_ci": [
        -0.0488,
        0.0346
      ],
      "loading_std": 0.2885,
      "implied_r": 0.0213,
      "dev": -0.0284,
      "dev_ci": [
        -0.0657,
        0.0103
      ],
      "dev_adj": -0.0152,
      "dev_adj_ci": [
        -0.0526,
        0.0226
      ]
    },
    "O5": {
      "r": 0.0799,
      "r_ci": [
        0.0383,
        0.1212
      ],
      "loading_std": 0.5467,
      "implied_r": 0.0403,
      "dev": 0.0396,
      "dev_ci": [
        0.008,
        0.0688
      ],
      "dev_adj": 0.0251,
      "dev_adj_ci": [
        -0.0033,
        0.0552
      ]
    }
  },
  "keying_contrast": {
    "unadjusted": {
      "d": 0.0474,
      "ci": [
        0.0292,
        0.0668
      ]
    },
    "adjusted": {
      "d": 0.0204,
      "ci": [
        0.0072,
        0.0348
      ]
    }
  },
  "prediction": {
    "A_unit": {
      "r2": 0.0224,
      "ci": [
        0.0092,
        0.0339
      ]
    },
    "A_factor": {
      "r2": 0.0138,
      "ci": [
        0.0033,
        0.0231
      ]
    },
    "A_items": {
      "r2": 0.0301,
      "ci": [
        0.0138,
        0.0438
      ]
    },
    "C_unit": {
      "r2": 0.0054,
      "ci": [
        -0.0032,
        0.0123
      ]
    },
    "C_factor": {
      "r2": 0.0062,
      "ci": [
        -0.003,
        0.0135
      ]
    },
    "C_items": {
      "r2": 0.0135,
      "ci": [
        0.0006,
        0.0245
      ]
    },
    "E_unit": {
      "r2": 0.0024,
      "ci": [
        -0.0038,
        0.0076
      ]
    },
    "E_factor": {
      "r2": 0.0023,
      "ci": [
        -0.0038,
        0.0076
      ]
    },
    "E_items": {
      "r2": 0.0224,
      "ci": [
        0.0085,
        0.0354
      ]
    },
    "N_unit": {
      "r2": 0.0074,
      "ci": [
        -0.0017,
        0.0157
      ]
    },
    "N_factor": {
      "r2": 0.007,
      "ci": [
        -0.0016,
        0.0149
      ]
    },
    "N_items": {
      "r2": 0.0127,
      "ci": [
        0.001,
        0.0229
      ]
    },
    "O_unit": {
      "r2": 0.0009,
      "ci": [
        -0.0046,
        0.0053
      ]
    },
    "O_factor": {
      "r2": 0.0014,
      "ci": [
        -0.0043,
        0.0061
      ]
    },
    "O_items": {
      "r2": 0.0008,
      "ci": [
        -0.0072,
        0.0074
      ]
    },
    "all_unit": {
      "r2": 0.0261,
      "ci": [
        0.0104,
        0.0401
      ]
    },
    "all_factor": {
      "r2": 0.0188,
      "ci": [
        0.0046,
        0.0312
      ]
    },
    "all_items": {
      "r2": 0.067,
      "ci": [
        0.0426,
        0.0893
      ]
    },
    "A_items_minus_factor": {
      "d": 0.0163,
      "ci": [
        0.0052,
        0.0271
      ]
    },
    "A_factor_minus_unit": {
      "d": -0.0086,
      "ci": [
        -0.0121,
        -0.0048
      ]
    },
    "C_items_minus_factor": {
      "d": 0.0073,
      "ci": [
        -0.0017,
        0.0159
      ]
    },
    "C_factor_minus_unit": {
      "d": 0.0008,
      "ci": [
        0,
        0.0016
      ]
    },
    "E_items_minus_factor": {
      "d": 0.0201,
      "ci": [
        0.0072,
        0.0329
      ]
    },
    "E_factor_minus_unit": {
      "d": -0.0001,
      "ci": [
        -0.0007,
        0.0006
      ]
    },
    "N_items_minus_factor": {
      "d": 0.0057,
      "ci": [
        -0.0022,
        0.0134
      ]
    },
    "N_factor_minus_unit": {
      "d": -0.0004,
      "ci": [
        -0.0024,
        0.0017
      ]
    },
    "O_items_minus_factor": {
      "d": -0.0007,
      "ci": [
        -0.0059,
        0.0043
      ]
    },
    "O_factor_minus_unit": {
      "d": 0.0005,
      "ci": [
        -0.0006,
        0.0015
      ]
    },
    "all_items_minus_factor": {
      "d": 0.0482,
      "ci": [
        0.028,
        0.0672
      ]
    },
    "all_factor_minus_unit": {
      "d": -0.0073,
      "ci": [
        -0.0111,
        -0.0033
      ]
    },
    "all_items_minus_unit": {
      "d": 0.0409,
      "ci": [
        0.0217,
        0.0597
      ]
    },
    "share_all_unit_of_items": {
      "ratio": 0.3892,
      "ci": [
        0.1829,
        0.5818
      ]
    },
    "share_all_factor_of_items": {
      "ratio": 0.2801,
      "ci": [
        0.0798,
        0.4578
      ]
    },
    "all_unit_r_pred_obs": 0.1624,
    "all_items_r_pred_obs": 0.2625
  },
  "replication": {
    "median_cross_half_r": 0.7769,
    "range_2_5_97_5": [
      0.6536,
      0.8866
    ],
    "null_median": -0.0266,
    "null_p95": 0.3674,
    "spearman_brown_full_sample": 0.8745,
    "share_splits_above_null_p95": 1,
    "splits": 100
  },
  "power": {
    "ncp_80pct_alpha05_df4": 11.9353,
    "ncp_80pct_alpha01_df4": 16.7493,
    "min_detectable_r_alpha05_power80": 0.0596
  },
  "decision": {
    "n_domains_rejected_holm": 4,
    "n_domains_items_beat_factor_ci": 2,
    "replication_exceeds_null_p95": true,
    "n_items_dev_ci_excludes_0": 13,
    "n_items_dev_adj_ci_excludes_0": 13
  },
  "bootstrap": {
    "B_null": 1999,
    "B_ci": 999,
    "B_robustness": 499,
    "B_prediction": 2000,
    "cv": "10-fold x 10",
    "seed": 20261024
  },
  "revision": {
    "mimic_fit_original": {
      "A": {
        "h0": {
          "chi2": 120.5913,
          "df": 9,
          "cfi": 0.9489,
          "rmsea": 0.0749,
          "srmr": 0.0371
        },
        "ha": {
          "chi2": 75.4393,
          "df": 5,
          "cfi": 0.9677,
          "rmsea": 0.0799,
          "srmr": 0.0275
        }
      },
      "C": {
        "h0": {
          "chi2": 151.2491,
          "df": 9,
          "cfi": 0.9339,
          "rmsea": 0.0846,
          "srmr": 0.0385
        },
        "ha": {
          "chi2": 127.4341,
          "df": 5,
          "cfi": 0.9431,
          "rmsea": 0.1053,
          "srmr": 0.0342
        }
      },
      "E": {
        "h0": {
          "chi2": 117.2187,
          "df": 9,
          "cfi": 0.9581,
          "rmsea": 0.0738,
          "srmr": 0.0347
        },
        "ha": {
          "chi2": 63.624,
          "df": 5,
          "cfi": 0.9773,
          "rmsea": 0.0729,
          "srmr": 0.0233
        }
      },
      "N": {
        "h0": {
          "chi2": 331.9998,
          "df": 9,
          "cfi": 0.9193,
          "rmsea": 0.1275,
          "srmr": 0.0509
        },
        "ha": {
          "chi2": 314.675,
          "df": 5,
          "cfi": 0.9227,
          "rmsea": 0.1675,
          "srmr": 0.0488
        }
      },
      "O": {
        "h0": {
          "chi2": 75.9938,
          "df": 9,
          "cfi": 0.944,
          "rmsea": 0.0581,
          "srmr": 0.0303
        },
        "ha": {
          "chi2": 67.6236,
          "df": 5,
          "cfi": 0.9477,
          "rmsea": 0.0753,
          "srmr": 0.0281
        }
      }
    },
    "acceptable_fit": {
      "A": {
        "test": {
          "chi2": 45.152,
          "df": 4,
          "p_asymptotic": 3.696822893098304e-9,
          "p_bootstrap": 0.002
        },
        "nps": 0.2326,
        "nps_ci": [
          0.1193,
          0.4524
        ],
        "nps_null_p95": 0.087,
        "loading_std": [
          0.361,
          0.6555,
          0.7676,
          0.4873,
          0.625
        ],
        "mimic_h0": {
          "chi2": 120.5913,
          "df": 9,
          "cfi": 0.9489,
          "rmsea": 0.0749,
          "srmr": 0.0371
        },
        "mimic_ha": {
          "chi2": 75.4393,
          "df": 5,
          "cfi": 0.9677,
          "rmsea": 0.0799,
          "srmr": 0.0275
        },
        "freed_pairs": [],
        "freed_residual_cov_std": [],
        "measurement_chi2": 75.4393,
        "measurement_df": 5,
        "measurement_rmsea": 0.0799,
        "rmsea_path": [
          0.0799
        ],
        "dev": {
          "A1": 0.0841,
          "A2": 0.0019,
          "A3": -0.0697,
          "A4": 0.0437,
          "A5": 0.0016
        },
        "p_holm": 0.01
      },
      "C": {
        "test": {
          "chi2": 22.0885,
          "df": 4,
          "p_asymptotic": 0.0001924542095608422,
          "p_bootstrap": 0.004
        },
        "nps": 0.2278,
        "nps_ci": [
          0.0921,
          0.636
        ],
        "nps_null_p95": 0.1819,
        "loading_std": [
          0.4778,
          0.5474,
          0.5502,
          0.7195,
          0.6258
        ],
        "mimic_h0": {
          "chi2": 75.4219,
          "df": 8,
          "cfi": 0.9687,
          "rmsea": 0.0618,
          "srmr": 0.0288
        },
        "mimic_ha": {
          "chi2": 53.3334,
          "df": 4,
          "cfi": 0.9771,
          "rmsea": 0.0747,
          "srmr": 0.0235
        },
        "freed_pairs": [
          [
            "C1",
            "C2"
          ]
        ],
        "freed_residual_cov_std": [
          0.1624
        ],
        "measurement_chi2": 53.3334,
        "measurement_df": 4,
        "measurement_rmsea": 0.0747,
        "rmsea_path": [
          0.1053,
          0.0747
        ],
        "dev": {
          "C1": 0.0216,
          "C2": -0.0592,
          "C3": -0.006,
          "C4": 0.0413,
          "C5": -0.0065
        },
        "p_holm": 0.01
      },
      "E": {
        "test": {
          "chi2": 53.5947,
          "df": 4,
          "p_asymptotic": 6.398326460794592e-11,
          "p_bootstrap": 0.002
        },
        "nps": 0.5669,
        "nps_ci": [
          0.2844,
          0.912
        ],
        "nps_null_p95": 0.2926,
        "loading_std": [
          0.5936,
          0.7374,
          0.6003,
          0.6937,
          0.5312
        ],
        "mimic_h0": {
          "chi2": 117.2187,
          "df": 9,
          "cfi": 0.9581,
          "rmsea": 0.0738,
          "srmr": 0.0347
        },
        "mimic_ha": {
          "chi2": 63.624,
          "df": 5,
          "cfi": 0.9773,
          "rmsea": 0.0729,
          "srmr": 0.0233
        },
        "freed_pairs": [],
        "freed_residual_cov_std": [],
        "measurement_chi2": 63.624,
        "measurement_df": 5,
        "measurement_rmsea": 0.0729,
        "rmsea_path": [
          0.0729
        ],
        "dev": {
          "E1": 0.0097,
          "E2": 0.0552,
          "E3": -0.0647,
          "E4": -0.0563,
          "E5": 0.0599
        },
        "p_holm": 0.01
      },
      "N": {
        "test": {
          "chi2": 22.8341,
          "df": 4,
          "p_asymptotic": 0.00013666511014678786,
          "p_bootstrap": 0.002
        },
        "nps": 0.1741,
        "nps_ci": [
          0.0657,
          0.476
        ],
        "nps_null_p95": 0.1472,
        "loading_std": [
          0.86,
          0.8277,
          0.6585,
          0.4713,
          0.4275
        ],
        "mimic_h0": {
          "chi2": 23.9916,
          "df": 6,
          "cfi": 0.9955,
          "rmsea": 0.0369,
          "srmr": 0.0166
        },
        "mimic_ha": {
          "chi2": 1.1575,
          "df": 2,
          "cfi": 1,
          "rmsea": 0,
          "srmr": 0.0022
        },
        "freed_pairs": [
          [
            "N4",
            "N5"
          ],
          [
            "N3",
            "N4"
          ],
          [
            "N3",
            "N5"
          ]
        ],
        "freed_residual_cov_std": [
          0.1994,
          0.214,
          0.1484
        ],
        "measurement_chi2": 1.1575,
        "measurement_df": 2,
        "measurement_rmsea": 0,
        "rmsea_path": [
          0.1675,
          0.1616,
          0.1149,
          0
        ],
        "dev": {
          "N1": 0.0196,
          "N2": 0.0087,
          "N3": -0.0292,
          "N4": 0.0383,
          "N5": -0.0537
        },
        "p_holm": 0.01
      },
      "O": {
        "test": {
          "chi2": 8.3703,
          "df": 4,
          "p_asymptotic": 0.07891789772167628,
          "p_bootstrap": 0.07
        },
        "nps": 0.2801,
        "nps_ci": [
          0.1081,
          0.8377
        ],
        "nps_null_p95": 0.5342,
        "loading_std": [
          0.541,
          0.4954,
          0.6325,
          0.2885,
          0.5467
        ],
        "mimic_h0": {
          "chi2": 75.9938,
          "df": 9,
          "cfi": 0.944,
          "rmsea": 0.0581,
          "srmr": 0.0303
        },
        "mimic_ha": {
          "chi2": 67.6236,
          "df": 5,
          "cfi": 0.9477,
          "rmsea": 0.0753,
          "srmr": 0.0281
        },
        "freed_pairs": [],
        "freed_residual_cov_std": [],
        "measurement_chi2": 67.6236,
        "measurement_df": 5,
        "measurement_rmsea": 0.0753,
        "rmsea_path": [
          0.0753
        ],
        "dev": {
          "O1": 0.0009,
          "O2": -0.0021,
          "O3": -0.0203,
          "O4": -0.0284,
          "O5": 0.0396
        },
        "p_holm": 0.07
      }
    },
    "positive_keyed": {
      "A": {
        "test": {
          "chi2": 25.2158,
          "df": 3,
          "p_asymptotic": 0.000013916446724623991,
          "p_bootstrap": 0.002
        },
        "nps": 0.163,
        "nps_ci": [
          0.0643,
          0.3837
        ],
        "nps_null_p95": 0.09,
        "loading_std": [
          0.6289,
          0.7794,
          0.4915,
          0.6361
        ],
        "mimic_h0": {
          "chi2": 35.0291,
          "df": 5,
          "cfi": 0.9838,
          "rmsea": 0.0522,
          "srmr": 0.0231
        },
        "mimic_ha": {
          "chi2": 9.8133,
          "df": 2,
          "cfi": 0.9958,
          "rmsea": 0.0421,
          "srmr": 0.0106
        },
        "items": [
          "A2",
          "A3",
          "A4",
          "A5"
        ],
        "dev": {
          "A2": 0.018,
          "A3": -0.0568,
          "A4": 0.0523,
          "A5": 0.0118
        },
        "p_holm": 0.01
      },
      "C": {
        "test": {
          "chi2": 13.2846,
          "df": 2,
          "p_asymptotic": 0.0013040048397837628,
          "p_bootstrap": 0.006
        },
        "nps": 0.4448,
        "nps_ci": [
          0.108,
          0.9815
        ],
        "nps_null_p95": 0.7607,
        "loading_std": [
          0.6125,
          0.6923,
          0.5122
        ],
        "mimic_h0": {
          "chi2": 13.2846,
          "df": 2,
          "cfi": 0.9864,
          "rmsea": 0.0506,
          "srmr": 0.0187
        },
        "mimic_ha": {
          "chi2": 0,
          "df": 0,
          "cfi": 1,
          "rmsea": null,
          "srmr": 0
        },
        "items": [
          "C1",
          "C2",
          "C3"
        ],
        "dev": {
          "C1": 0.0338,
          "C2": -0.0447,
          "C3": 0.0201
        },
        "p_holm": 0.012
      },
      "E": {
        "test": {
          "chi2": 31.3193,
          "df": 2,
          "p_asymptotic": 1.5816415896726245e-7,
          "p_bootstrap": 0.002
        },
        "nps": 0.8908,
        "nps_ci": [
          0.4938,
          0.9995
        ],
        "nps_null_p95": 0.9901,
        "loading_std": [
          0.7242,
          0.5923,
          0.5561
        ],
        "mimic_h0": {
          "chi2": 31.3193,
          "df": 2,
          "cfi": 0.9687,
          "rmsea": 0.0815,
          "srmr": 0.0303
        },
        "mimic_ha": {
          "chi2": 0,
          "df": 0,
          "cfi": 1,
          "rmsea": null,
          "srmr": 0
        },
        "items": [
          "E3",
          "E4",
          "E5"
        ],
        "dev": {
          "E3": -0.043,
          "E4": -0.0237,
          "E5": 0.0816
        },
        "p_holm": 0.01
      },
      "N": {
        "test": {
          "chi2": 17.3248,
          "df": 4,
          "p_asymptotic": 0.0016712930202802071,
          "p_bootstrap": 0.004
        },
        "nps": 0.1594,
        "nps_ci": [
          0.0537,
          0.4738
        ],
        "nps_null_p95": 0.1131,
        "loading_std": [
          0.825,
          0.8033,
          0.7202,
          0.5563,
          0.4959
        ],
        "mimic_h0": {
          "chi2": 331.9998,
          "df": 9,
          "cfi": 0.9193,
          "rmsea": 0.1275,
          "srmr": 0.0509
        },
        "mimic_ha": {
          "chi2": 314.675,
          "df": 5,
          "cfi": 0.9227,
          "rmsea": 0.1675,
          "srmr": 0.0488
        },
        "items": [
          "N1",
          "N2",
          "N3",
          "N4",
          "N5"
        ],
        "dev": {
          "N1": 0.014,
          "N2": 0.0042,
          "N3": -0.0241,
          "N4": 0.0463,
          "N5": -0.0473
        },
        "p_holm": 0.012
      },
      "O": {
        "test": {
          "chi2": 1.4696,
          "df": 2,
          "p_asymptotic": 0.4796126282438351,
          "p_bootstrap": 0.456
        },
        "nps": 0.2297,
        "nps_ci": [
          0.0163,
          0.9894
        ],
        "nps_null_p95": 0.9196,
        "loading_std": [
          0.6324,
          0.609,
          0.2878
        ],
        "mimic_h0": {
          "chi2": 1.4696,
          "df": 2,
          "cfi": 1,
          "rmsea": 0,
          "srmr": 0.0074
        },
        "mimic_ha": {
          "chi2": 0,
          "df": 0,
          "cfi": 1,
          "rmsea": null,
          "srmr": 0
        },
        "items": [
          "O1",
          "O3",
          "O4"
        ],
        "dev": {
          "O1": 0.0113,
          "O3": -0.002,
          "O4": -0.0205
        },
        "p_holm": 0.456
      }
    },
    "eq3": {
      "age_sd_decades": 1.0391,
      "age_mu3_decades3": 1.1221,
      "age_skewness": 1.0002,
      "domains": {
        "A": {
          "gamma_per_decade": 0.1402,
          "gamma_x_sd_age": 0.1457,
          "loading_drift_needed_d04": 0.2745,
          "loading_drift_needed_d08": 0.549
        },
        "C": {
          "gamma_per_decade": 0.1029,
          "gamma_x_sd_age": 0.1069,
          "loading_drift_needed_d04": 0.374,
          "loading_drift_needed_d08": 0.748
        },
        "E": {
          "gamma_per_decade": 0.0737,
          "gamma_x_sd_age": 0.0766,
          "loading_drift_needed_d04": 0.5224,
          "loading_drift_needed_d08": 1.0447
        },
        "N": {
          "gamma_per_decade": -0.1006,
          "gamma_x_sd_age": -0.1046,
          "loading_drift_needed_d04": 0.3825,
          "loading_drift_needed_d08": 0.765
        },
        "O": {
          "gamma_per_decade": 0.0715,
          "gamma_x_sd_age": 0.0743,
          "loading_drift_needed_d04": 0.5386,
          "loading_drift_needed_d08": 1.0771
        }
      }
    },
    "estimator_check": {
      "A": {
        "chi2_df4_custom": 45.152,
        "chi2_df4_independent": 45.152,
        "chi2_df5_custom": 75.4393,
        "chi2_df5_sklearn": 75.4393,
        "chi2_df5_independent": 75.4393
      },
      "C": {
        "chi2_df4_custom": 23.815,
        "chi2_df4_independent": 23.815,
        "chi2_df5_custom": 127.4341,
        "chi2_df5_sklearn": 127.4341,
        "chi2_df5_independent": 127.4341
      },
      "E": {
        "chi2_df4_custom": 53.5947,
        "chi2_df4_independent": 53.5947,
        "chi2_df5_custom": 63.624,
        "chi2_df5_sklearn": 63.624,
        "chi2_df5_independent": 63.624
      },
      "N": {
        "chi2_df4_custom": 17.3248,
        "chi2_df4_independent": 17.3248,
        "chi2_df5_custom": 314.675,
        "chi2_df5_sklearn": 314.675,
        "chi2_df5_independent": 314.675
      },
      "O": {
        "chi2_df4_custom": 8.3703,
        "chi2_df4_independent": 8.3703,
        "chi2_df5_custom": 67.6236,
        "chi2_df5_sklearn": 67.6236,
        "chi2_df5_independent": 67.6236
      },
      "max_abs_diff": {
        "loading_std_sklearn": 2.729731930495305e-7,
        "chi2_df5_sklearn": 3.884750299221196e-10,
        "chi2_df4_independent": 3.5119995800414467e-10,
        "chi2_df5_independent": 3.4138736282329774e-10,
        "loading_std_independent": 2.781398193985041e-7
      }
    },
    "acceptable_fit_n_rejected_holm": 4,
    "positive_keyed_n_rejected_holm": 4,
    "cu_fit": {
      "A": {
        "pair": [
          "A1",
          "A2"
        ],
        "test_chi2": 48.891,
        "measurement": {
          "chi2": 16.2106,
          "df": 4,
          "cfi": 0.9941,
          "rmsea": 0.0372,
          "srmr": 0.0141
        },
        "mimic_h0": {
          "chi2": 65.1016,
          "df": 8,
          "cfi": 0.9738,
          "rmsea": 0.0569,
          "srmr": 0.0296
        },
        "mimic_ha": {
          "chi2": 16.2106,
          "df": 4,
          "cfi": 0.9944,
          "rmsea": 0.0372,
          "srmr": 0.0118
        }
      },
      "C": {
        "pair": [
          "C1",
          "C2"
        ],
        "test_chi2": 22.0885,
        "measurement": {
          "chi2": 53.3334,
          "df": 4,
          "cfi": 0.9765,
          "rmsea": 0.0747,
          "srmr": 0.0279
        },
        "mimic_h0": {
          "chi2": 75.4219,
          "df": 8,
          "cfi": 0.9687,
          "rmsea": 0.0618,
          "srmr": 0.0288
        },
        "mimic_ha": {
          "chi2": 53.3334,
          "df": 4,
          "cfi": 0.9771,
          "rmsea": 0.0747,
          "srmr": 0.0235
        }
      },
      "E": {
        "pair": [
          "E3",
          "E5"
        ],
        "test_chi2": 57.5007,
        "measurement": {
          "chi2": 15.7687,
          "df": 4,
          "cfi": 0.9954,
          "rmsea": 0.0365,
          "srmr": 0.0118
        },
        "mimic_h0": {
          "chi2": 73.2694,
          "df": 8,
          "cfi": 0.9748,
          "rmsea": 0.0608,
          "srmr": 0.0281
        },
        "mimic_ha": {
          "chi2": 15.7687,
          "df": 4,
          "cfi": 0.9954,
          "rmsea": 0.0365,
          "srmr": 0.01
        }
      },
      "N": {
        "pair": [
          "N4",
          "N5"
        ],
        "test_chi2": 20.6722,
        "measurement": {
          "chi2": 234.6191,
          "df": 4,
          "cfi": 0.9414,
          "rmsea": 0.1616,
          "srmr": 0.0513
        },
        "mimic_h0": {
          "chi2": 255.2913,
          "df": 8,
          "cfi": 0.9383,
          "rmsea": 0.1183,
          "srmr": 0.0458
        },
        "mimic_ha": {
          "chi2": 234.6191,
          "df": 4,
          "cfi": 0.9424,
          "rmsea": 0.1616,
          "srmr": 0.0431
        }
      },
      "O": {
        "pair": [
          "O2",
          "O5"
        ],
        "test_chi2": 9.7392,
        "measurement": {
          "chi2": 21.6682,
          "df": 4,
          "cfi": 0.985,
          "rmsea": 0.0447,
          "srmr": 0.0194
        },
        "mimic_h0": {
          "chi2": 31.4074,
          "df": 8,
          "cfi": 0.9804,
          "rmsea": 0.0364,
          "srmr": 0.0205
        },
        "mimic_ha": {
          "chi2": 21.6682,
          "df": 4,
          "cfi": 0.9852,
          "rmsea": 0.0447,
          "srmr": 0.0164
        }
      }
    },
    "adjusted_fit": {
      "A": {
        "chi2": 58.05,
        "df": 5,
        "cfi": 0.9718,
        "rmsea": 0.0693,
        "srmr": 0.0296,
        "test_chi2": 39.1968
      },
      "C": {
        "chi2": 104.0452,
        "df": 5,
        "cfi": 0.9533,
        "rmsea": 0.0947,
        "srmr": 0.0369,
        "test_chi2": 15.9685
      },
      "E": {
        "chi2": 53.2572,
        "df": 5,
        "cfi": 0.9805,
        "rmsea": 0.0661,
        "srmr": 0.0258,
        "test_chi2": 42.8367
      },
      "N": {
        "chi2": 312.9374,
        "df": 5,
        "cfi": 0.9156,
        "rmsea": 0.167,
        "srmr": 0.0617,
        "test_chi2": 16.3534
      },
      "O": {
        "chi2": 44.3136,
        "df": 5,
        "cfi": 0.966,
        "rmsea": 0.0597,
        "srmr": 0.0268,
        "test_chi2": 9.8861
      }
    },
    "fit_reproduces_table1": true,
    "item_bands": {
      "unadjusted": {
        "critical_value": 3.0467,
        "critical_value_mc_95": [
          2.9915,
          3.1195
        ],
        "share_resamples_inside_band": 0.9499,
        "items": {
          "A1": {
            "dev": 0.0841,
            "boot_sd": 0.0183,
            "band": [
              0.0283,
              0.1398
            ],
            "p_maxT": 0.002,
            "p_holm": 0.00010464478129531036
          },
          "A2": {
            "dev": 0.0019,
            "boot_sd": 0.0139,
            "band": [
              -0.0406,
              0.0444
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "A3": {
            "dev": -0.0697,
            "boot_sd": 0.0126,
            "band": [
              -0.1082,
              -0.0312
            ],
            "p_maxT": 0.001,
            "p_holm": 8.910667917532407e-7
          },
          "A4": {
            "dev": 0.0437,
            "boot_sd": 0.0169,
            "band": [
              -0.0079,
              0.0953
            ],
            "p_maxT": 0.192,
            "p_holm": 0.14753319188375175
          },
          "A5": {
            "dev": 0.0016,
            "boot_sd": 0.0148,
            "band": [
              -0.0434,
              0.0466
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "C1": {
            "dev": 0.0167,
            "boot_sd": 0.0174,
            "band": [
              -0.0363,
              0.0696
            ],
            "p_maxT": 0.999,
            "p_holm": 1
          },
          "C2": {
            "dev": -0.0632,
            "boot_sd": 0.0168,
            "band": [
              -0.1144,
              -0.0121
            ],
            "p_maxT": 0.004,
            "p_holm": 0.0032624048136013096
          },
          "C3": {
            "dev": -0.0046,
            "boot_sd": 0.0151,
            "band": [
              -0.0506,
              0.0415
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "C4": {
            "dev": 0.0474,
            "boot_sd": 0.0133,
            "band": [
              0.0069,
              0.088
            ],
            "p_maxT": 0.004,
            "p_holm": 0.00688164624316448
          },
          "C5": {
            "dev": -0.0007,
            "boot_sd": 0.0154,
            "band": [
              -0.0477,
              0.0462
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "E1": {
            "dev": 0.0097,
            "boot_sd": 0.0147,
            "band": [
              -0.0352,
              0.0546
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "E2": {
            "dev": 0.0552,
            "boot_sd": 0.0132,
            "band": [
              0.0149,
              0.0955
            ],
            "p_maxT": 0.004,
            "p_holm": 0.000674020616778235
          },
          "E3": {
            "dev": -0.0647,
            "boot_sd": 0.0153,
            "band": [
              -0.1111,
              -0.0182
            ],
            "p_maxT": 0.003,
            "p_holm": 0.0005150530268665834
          },
          "E4": {
            "dev": -0.0563,
            "boot_sd": 0.0145,
            "band": [
              -0.1005,
              -0.0121
            ],
            "p_maxT": 0.004,
            "p_holm": 0.0021867764747462316
          },
          "E5": {
            "dev": 0.0599,
            "boot_sd": 0.0173,
            "band": [
              0.0072,
              0.1126
            ],
            "p_maxT": 0.011,
            "p_holm": 0.009540198750581744
          },
          "N1": {
            "dev": 0.014,
            "boot_sd": 0.0116,
            "band": [
              -0.0214,
              0.0493
            ],
            "p_maxT": 0.996,
            "p_holm": 1
          },
          "N2": {
            "dev": 0.0042,
            "boot_sd": 0.0131,
            "band": [
              -0.0358,
              0.0442
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "N3": {
            "dev": -0.0241,
            "boot_sd": 0.0119,
            "band": [
              -0.0603,
              0.0121
            ],
            "p_maxT": 0.619,
            "p_holm": 0.5498453203712795
          },
          "N4": {
            "dev": 0.0463,
            "boot_sd": 0.0157,
            "band": [
              -0.0016,
              0.0942
            ],
            "p_maxT": 0.075,
            "p_holm": 0.05472129640441154
          },
          "N5": {
            "dev": -0.0473,
            "boot_sd": 0.0173,
            "band": [
              -0.0999,
              0.0053
            ],
            "p_maxT": 0.135,
            "p_holm": 0.09827530263637912
          },
          "O1": {
            "dev": 0.0009,
            "boot_sd": 0.0162,
            "band": [
              -0.0485,
              0.0503
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "O2": {
            "dev": -0.0021,
            "boot_sd": 0.0164,
            "band": [
              -0.0519,
              0.0477
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "O3": {
            "dev": -0.0203,
            "boot_sd": 0.0145,
            "band": [
              -0.0644,
              0.0237
            ],
            "p_maxT": 0.973,
            "p_holm": 1
          },
          "O4": {
            "dev": -0.0284,
            "boot_sd": 0.0187,
            "band": [
              -0.0854,
              0.0286
            ],
            "p_maxT": 0.951,
            "p_holm": 1
          },
          "O5": {
            "dev": 0.0396,
            "boot_sd": 0.0154,
            "band": [
              -0.0073,
              0.0865
            ],
            "p_maxT": 0.195,
            "p_holm": 0.14753319188375175
          }
        },
        "n_band_excludes_0": 8,
        "n_holm_below_05": 8,
        "band_excludes_0": [
          "A1",
          "A3",
          "C2",
          "C4",
          "E2",
          "E3",
          "E4",
          "E5"
        ],
        "holm_below_05": [
          "A1",
          "A3",
          "C2",
          "C4",
          "E2",
          "E3",
          "E4",
          "E5"
        ]
      },
      "adjusted": {
        "critical_value": 3.1125,
        "critical_value_mc_95": [
          3.0017,
          3.2038
        ],
        "share_resamples_inside_band": 0.9499,
        "items": {
          "A1": {
            "dev": 0.0295,
            "boot_sd": 0.0184,
            "band": [
              -0.0277,
              0.0868
            ],
            "p_maxT": 0.909,
            "p_holm": 1
          },
          "A2": {
            "dev": -0.0024,
            "boot_sd": 0.0142,
            "band": [
              -0.0464,
              0.0417
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "A3": {
            "dev": -0.0662,
            "boot_sd": 0.013,
            "band": [
              -0.1065,
              -0.0259
            ],
            "p_maxT": 0.001,
            "p_holm": 0.000008002726469198023
          },
          "A4": {
            "dev": 0.0803,
            "boot_sd": 0.0172,
            "band": [
              0.0267,
              0.1339
            ],
            "p_maxT": 0.002,
            "p_holm": 0.00007581461865886683
          },
          "A5": {
            "dev": 0.0063,
            "boot_sd": 0.015,
            "band": [
              -0.0403,
              0.053
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "C1": {
            "dev": 0.0139,
            "boot_sd": 0.0173,
            "band": [
              -0.0401,
              0.0678
            ],
            "p_maxT": 0.999,
            "p_holm": 1
          },
          "C2": {
            "dev": -0.0543,
            "boot_sd": 0.0171,
            "band": [
              -0.1077,
              -0.001
            ],
            "p_maxT": 0.042,
            "p_holm": 0.030272088433299805
          },
          "C3": {
            "dev": -0.012,
            "boot_sd": 0.0151,
            "band": [
              -0.0591,
              0.0352
            ],
            "p_maxT": 0.999,
            "p_holm": 1
          },
          "C4": {
            "dev": 0.0297,
            "boot_sd": 0.0132,
            "band": [
              -0.0113,
              0.0707
            ],
            "p_maxT": 0.415,
            "p_holm": 0.361843574615384
          },
          "C5": {
            "dev": 0.0186,
            "boot_sd": 0.0154,
            "band": [
              -0.0293,
              0.0666
            ],
            "p_maxT": 0.995,
            "p_holm": 1
          },
          "E1": {
            "dev": -0.0077,
            "boot_sd": 0.0155,
            "band": [
              -0.0558,
              0.0405
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "E2": {
            "dev": 0.0484,
            "boot_sd": 0.0129,
            "band": [
              0.0081,
              0.0887
            ],
            "p_maxT": 0.003,
            "p_holm": 0.004028793100124844
          },
          "E3": {
            "dev": -0.0656,
            "boot_sd": 0.0156,
            "band": [
              -0.114,
              -0.0171
            ],
            "p_maxT": 0.003,
            "p_holm": 0.0005817844463804362
          },
          "E4": {
            "dev": -0.0359,
            "boot_sd": 0.015,
            "band": [
              -0.0827,
              0.0109
            ],
            "p_maxT": 0.333,
            "p_holm": 0.28052334578824945
          },
          "E5": {
            "dev": 0.0613,
            "boot_sd": 0.018,
            "band": [
              0.0052,
              0.1173
            ],
            "p_maxT": 0.021,
            "p_holm": 0.013981876461363917
          },
          "N1": {
            "dev": 0.0237,
            "boot_sd": 0.0116,
            "band": [
              -0.0123,
              0.0596
            ],
            "p_maxT": 0.581,
            "p_holm": 0.5672940169294237
          },
          "N2": {
            "dev": 0.0043,
            "boot_sd": 0.0132,
            "band": [
              -0.0369,
              0.0455
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "N3": {
            "dev": -0.0295,
            "boot_sd": 0.0123,
            "band": [
              -0.0678,
              0.0088
            ],
            "p_maxT": 0.327,
            "p_holm": 0.28052334578824945
          },
          "N4": {
            "dev": 0.0335,
            "boot_sd": 0.0165,
            "band": [
              -0.0177,
              0.0848
            ],
            "p_maxT": 0.594,
            "p_holm": 0.5672940169294237
          },
          "N5": {
            "dev": -0.0449,
            "boot_sd": 0.0181,
            "band": [
              -0.1013,
              0.0115
            ],
            "p_maxT": 0.268,
            "p_holm": 0.25281187119645093
          },
          "O1": {
            "dev": 0.0295,
            "boot_sd": 0.0165,
            "band": [
              -0.022,
              0.0809
            ],
            "p_maxT": 0.802,
            "p_holm": 0.8946550441003491
          },
          "O2": {
            "dev": -0.038,
            "boot_sd": 0.0157,
            "band": [
              -0.0867,
              0.0107
            ],
            "p_maxT": 0.298,
            "p_holm": 0.2725287215845357
          },
          "O3": {
            "dev": -0.0094,
            "boot_sd": 0.0149,
            "band": [
              -0.0559,
              0.0371
            ],
            "p_maxT": 1,
            "p_holm": 1
          },
          "O4": {
            "dev": -0.0152,
            "boot_sd": 0.0189,
            "band": [
              -0.0739,
              0.0435
            ],
            "p_maxT": 0.999,
            "p_holm": 1
          },
          "O5": {
            "dev": 0.0251,
            "boot_sd": 0.015,
            "band": [
              -0.0215,
              0.0718
            ],
            "p_maxT": 0.864,
            "p_holm": 1
          }
        },
        "n_band_excludes_0": 6,
        "n_holm_below_05": 6,
        "band_excludes_0": [
          "A3",
          "A4",
          "C2",
          "E2",
          "E3",
          "E5"
        ],
        "holm_below_05": [
          "A3",
          "A4",
          "C2",
          "E2",
          "E3",
          "E5"
        ]
      }
    },
    "fiml": {
      "n_age_eligible": 2543,
      "n_incomplete_any_item": 334,
      "domains": {
        "A": {
          "n_used": 2543,
          "n_with_missing_items_used": 81,
          "n_patterns": 12,
          "chi2": 54.3304,
          "df": 4,
          "p_asymptotic": 4.4876761298695845e-11,
          "converged": true,
          "complete_case_chi2": 45.152,
          "complete_case_p_asymptotic": 3.696822893098304e-9,
          "p_holm_asymptotic": 1.7950704519478338e-10,
          "complete_case_p_holm_asymptotic": 1.4787291572393217e-8
        },
        "C": {
          "n_used": 2543,
          "n_with_missing_items_used": 82,
          "n_patterns": 11,
          "chi2": 23.6915,
          "df": 4,
          "p_asymptotic": 0.00009209245104349472,
          "converged": true,
          "complete_case_chi2": 23.815,
          "complete_case_p_asymptotic": 0.00008699322810840304,
          "p_holm_asymptotic": 0.0002762773531304842,
          "complete_case_p_holm_asymptotic": 0.0002609796843252091
        },
        "E": {
          "n_used": 2543,
          "n_with_missing_items_used": 79,
          "n_patterns": 7,
          "chi2": 67.6705,
          "df": 4,
          "p_asymptotic": 7.039878686676016e-14,
          "converged": true,
          "complete_case_chi2": 53.5947,
          "complete_case_p_asymptotic": 6.398326460794592e-11,
          "p_holm_asymptotic": 3.519939343338008e-13,
          "complete_case_p_holm_asymptotic": 3.199163230397297e-10
        },
        "N": {
          "n_used": 2543,
          "n_with_missing_items_used": 95,
          "n_patterns": 11,
          "chi2": 17.4124,
          "df": 4,
          "p_asymptotic": 0.0016069338239945354,
          "converged": true,
          "complete_case_chi2": 17.3248,
          "complete_case_p_asymptotic": 0.0016712930202802071,
          "p_holm_asymptotic": 0.003213867647989071,
          "complete_case_p_holm_asymptotic": 0.0033425860405604147
        },
        "O": {
          "n_used": 2543,
          "n_with_missing_items_used": 61,
          "n_patterns": 7,
          "chi2": 12.5455,
          "df": 4,
          "p_asymptotic": 0.013723645330854793,
          "converged": true,
          "complete_case_chi2": 8.3703,
          "complete_case_p_asymptotic": 0.07891789772167628,
          "p_holm_asymptotic": 0.013723645330854793,
          "complete_case_p_holm_asymptotic": 0.07891789772167628
        }
      },
      "n_rejected_holm": 5,
      "complete_case_n_rejected_holm_asymptotic": 4,
      "rejected": [
        "A",
        "C",
        "E",
        "N",
        "O"
      ],
      "complete_case_reproduction_max_abs_diff": 3.8651393197142165e-10,
      "available_cases": {
        "domains": {
          "A": {
            "chi2": 53.9905,
            "df": 4,
            "p_asymptotic": 5.2868834733973636e-11,
            "p_bootstrap": 0.002,
            "n": 2462,
            "p_bootstrap_mc_se": 0.002,
            "p_holm": 0.01
          },
          "C": {
            "chi2": 24.0661,
            "df": 4,
            "p_asymptotic": 0.00007747493597429081,
            "p_bootstrap": 0.002,
            "n": 2461,
            "p_bootstrap_mc_se": 0.002,
            "p_holm": 0.01
          },
          "E": {
            "chi2": 65.5682,
            "df": 4,
            "p_asymptotic": 1.953206828655673e-13,
            "p_bootstrap": 0.002,
            "n": 2464,
            "p_bootstrap_mc_se": 0.002,
            "p_holm": 0.01
          },
          "N": {
            "chi2": 16.4059,
            "df": 4,
            "p_asymptotic": 0.0025201359939616692,
            "p_bootstrap": 0.004,
            "n": 2448,
            "p_bootstrap_mc_se": 0.0028,
            "p_holm": 0.01
          },
          "O": {
            "chi2": 11.7184,
            "df": 4,
            "p_asymptotic": 0.01957251908179676,
            "p_bootstrap": 0.024,
            "n": 2482,
            "p_bootstrap_mc_se": 0.0069,
            "p_holm": 0.024
          }
        },
        "n_rejected_holm": 5,
        "rejected": [
          "A",
          "C",
          "E",
          "N",
          "O"
        ]
      }
    }
  },
  "software": {
    "python": "3.12.13",
    "numpy": "2.5.2",
    "scipy": "1.18.1",
    "pandas": "3.0.5",
    "scikit_learn": "1.9.0"
  },
  "hypothesis_supported": true,
  "runtime_seconds": 49.4072
}
Experiment script · experiment_v1.py
"""Are age differences in personality domain-general or nuance-specific?  (real data: SAPA bfi, `from dataset import load`)
Executed design. Sample: ages 18-65 with all 25 items (complete cases). Items reverse-keyed 7 - x (A1 C4 C5 E1 E2 O2 O5);
domain score = mean of its five keyed items; six-point items treated as continuous, normal-theory ML, resampling inference.
H1 per domain: H0 (domain-general) x = nu + l*eta + e, eta = gamma*age + zeta, i.e. item-age associations proportional to the
loadings (VanderWeele & Vansteelandt, 2022; MIMIC without direct effects) versus HA with free item-specific age slopes and
the same one-factor residual structure. LR chi-square(4); p from a Bollen-Stine-type bootstrap (residuals rotated to the
H0-implied covariance, means regenerated from the H0 fit; 1,999 draws); Holm over five domains; supported if >= 4 rejected.
Effect sizes: non-proportional share NPS = 1 - cos^2(r, l) of item-age correlations r vs SD-scaled loadings l, with
nonparametric bootstrap CIs (999) and its H0 distribution; item deviations r - implied r; latent age correlation.
Sensitivity: one item's direct age effect detectable with 80% power at alpha = .01 (noncentral chi-square).
Robustness (499 bootstrap draws each): restricted cubic-spline age (4 knots, chi-square(12)); gender + education dummies +
leave-domain-out acquiescence partialled with free direct effects; one correlated uniqueness (largest residual pair);
loading-free rank-one test that the five items share one age curve (canonical correlations, chi-square(8)).
H2: 10-fold CV x 10 repeats predicting age by OLS from unit scores, training-fold Bartlett factor scores or items, per
domain and for all five domains; out-of-sample R^2 with 2,000 participant-bootstrap CIs. H3: 100 random half-splits,
cross-half correlation of the 25 item deviations versus an age-permutation null.
Revision analyses (added after peer review). R1-R7 draw no new random numbers - every bootstrap below reuses rows of the
same seeded resample matrix IDX - and R8b draws its resamples from a separate seeded stream, so all results above are
reproduced exactly:
 R1 fit of the H0 and HA MIMIC models themselves: joint covariance of the items and age; chi-square, CFI against the
    independence model of all items and age, RMSEA, SRMR;
 R2 acceptable-fit measurement models: residual covariances freed one pair at a time, largest absolute standardized
    residual first, until the HA measurement model reaches RMSEA <= .08 (at most 3 pairs); the H0-vs-HA test (499 Bollen-
    Stine draws), the NPS (499 nonparametric draws) and both models' fit are recomputed with H0 and HA sharing that
    structure, and the H1 rule (Holm over five domains, >= 4 rejected) is re-applied;
 R3 positively keyed items only (A2-A5, C1-C3, E3-E5, O1 O3 O4; N unchanged): the proportionality test with df = items - 1,
    499 Bollen-Stine draws, Holm over five domains;
 R4 moments behind the loading-drift bound (Eq. 3): if loadings drift linearly with age while intercepts do not, an item's
    deviation in the correlation metric is d = delta* x gamma*SD(age) x skew(age), where delta* is the drift of the
    SD-scaled loading per SD of age; reports age's third central moment and skewness, gamma per domain, and the delta*
    needed for deviations of .04 and .08;
 R5 estimator cross-check: HA loadings and chi-square(5) against sklearn FactorAnalysis (ML) on age-partialled items, and
    H0/HA chi-squares and loadings against an independent fit of the joint MIMIC covariance structure (different
    parameterisation, BFGS with numerical gradients, different start values);
 R6 fit (chi-square, CFI, RMSEA, SRMR) of the measurement structures used in two robustness checks: the model with the
    one correlated uniqueness freed (measurement model and both MIMIC models) and the covariate/acquiescence-adjusted model;
 R7 multiplicity-adjusted inference for the 25 item deviations from the same 999 nonparametric resamples: a simultaneous
    95% band (deviation +/- c x bootstrap SD, c the 95th percentile of the maximum |t| over the 25 items), single-step
    max-T adjusted p values, and Holm-adjusted p values from bootstrap z = deviation / bootstrap SD; also for the
    covariate/acquiescence-adjusted deviations;
 R8 missing data: (a) full-information ML of the primary H0-vs-HA test on ALL age-eligible participants (18-65, n = 2,543),
    conditional on age (observed for everyone; MAR given age and the observed items), asymptotic chi-square(4) p values
    with Holm over five domains, checked to reproduce the complete-case statistic when given the complete cases; (b) the
    primary test on the available cases of each domain (all five of its items observed), with B_ROB Bollen-Stine draws
    from the separate stream SeedSequence(20261024).spawn(1)[0] and Holm over five domains.
Seed 20261024."""
import json, platform, time
import numpy as np, pandas as pd, scipy, sklearn
from scipy import optimize, stats
from sklearn.decomposition import FactorAnalysis
from dataset import load

T0 = time.time()
rng = np.random.default_rng(20261024)
DOMS = {"A": "Agreeableness", "C": "Conscientiousness", "E": "Extraversion", "N": "Neuroticism", "O": "Openness"}
ITEMS = [f"{d}{i}" for d in DOMS for i in range(1, 6)]
REV = ["A1", "C4", "C5", "E1", "E2", "O2", "O5"]
BAL = {"A": ["A1", "A2"], "C": ["C1", "C2", "C4", "C5"], "E": ["E1", "E2", "E3", "E4"], "N": [], "O": ["O1", "O3", "O2", "O5"]}
B_NULL, B_NP, B_ROB, B_CV, SPLITS, REPS, FOLDS = 1999, 999, 499, 2000, 100, 10, 10
R4 = lambda v: round(float(v), 4)
CI = lambda a: [R4(np.percentile(a, 2.5)), R4(np.percentile(a, 97.5))]
cen = lambda A: A - A.mean(0)

# ------------------------------------------------------------------ data, inspection, sample flow
df = load()
print("columns:", list(df.columns), "| dtypes:", sorted(set(df.dtypes.astype(str))))
print("item NA per item: min", int(df[ITEMS].isna().sum().min()), "max", int(df[ITEMS].isna().sum().max()),
      "| education NA", int(df.education.isna().sum()), "| age", int(df.age.min()), "-", int(df.age.max()))
v = df[ITEMS].to_numpy(float)
assert set(np.unique(v[~np.isnan(v)]).tolist()) <= {1.0, 2.0, 3.0, 4.0, 5.0, 6.0} and set(df.gender.unique()) <= {1, 2}
in_age, complete = df.age.between(18, 65), df[ITEMS].notna().all(axis=1)
flow = {"n_total": int(len(df)), "excluded_age_under_18": int((df.age < 18).sum()), "excluded_age_over_65": int((df.age > 65).sum()),
        "excluded_incomplete_items_in_age_range": int((in_age & ~complete).sum())}
d = df[in_age & complete].reset_index(drop=True)
N = flow["n_analysed"] = len(d)
raw = d[ITEMS].to_numpy(float)
X = raw.copy()
for it in REV:
    X[:, ITEMS.index(it)] = 7 - X[:, ITEMS.index(it)]          # reverse-key: 7 - x
age, female = d.age.to_numpy(float), (d.gender == 2).to_numpy(float)
edu = d.education.fillna(0).astype(int).to_numpy()
EDU = np.column_stack([(edu == k) for k in (1, 2, 4, 5, 0)]).astype(float)  # reference 3 = some college; 0 = missing
cols = {dm: [ITEMS.index(f"{dm}{i}") for i in range(1, 6)] for dm in DOMS}
sample = {"age_mean": R4(age.mean()), "age_sd": R4(age.std(ddof=1)), "age_median": R4(np.median(age)), "n_age_18_29": int((age < 30).sum()),
          "n_age_30_44": int(((age >= 30) & (age < 45)).sum()), "n_age_45_65": int((age >= 45).sum()), "pct_female": R4(100 * female.mean()),
          "n_education_missing": int((edu == 0).sum()), "pct_edu_some_college": R4(100 * (edu == 3).mean()),
          "pct_edu_college_or_more": R4(100 * (edu >= 4).mean()), "mean_age_incomplete_excluded": R4(df.age[in_age & ~complete].mean())}

def acq(excl):  # leave-domain-out acquiescence: mean RAW response to balanced positively/negatively keyed items
    return raw[:, [ITEMS.index(i) for dm, its in BAL.items() if dm != excl for i in its]].mean(1)

def resid_on(A, W):  # partial nuisance covariates (free direct effects) out of items and age (Frisch-Waugh)
    W1 = np.column_stack([np.ones(len(W)), W])
    return cen(A - W1 @ np.linalg.lstsq(W1, A, rcond=None)[0])

def rcs(x, probs=(5, 35, 65, 95)):  # restricted cubic spline (Harrell, 2015): 4 knots -> 3 columns
    t, out = np.percentile(x, probs), [x]
    for j in range(2):
        out.append((np.clip(x - t[j], 0, None) ** 3 - np.clip(x - t[2], 0, None) ** 3 * (t[3] - t[j]) / (t[3] - t[2])
                    + np.clip(x - t[3], 0, None) ** 3 * (t[2] - t[j]) / (t[3] - t[2])) / (t[3] - t[0]) ** 2)
    Z = np.column_stack(out)
    return cen(Z / Z.std(0)), t

def msqrt(S, power):
    w, V = np.linalg.eigh(S)
    return (V * w ** power) @ V.T

# ------------------------------------------------------------------ ML one-factor models with analytic gradients
def pairs_of(cu):  # residual covariances: None, one (i, j) pair, or a list of pairs
    if cu is None or len(cu) == 0:
        return []
    return [tuple(cu)] if np.isscalar(cu[0]) else [tuple(c) for c in cu]

def sigma(th, cu, p=5):
    """Model covariance ll' + Psi (+ freed residual covariances, stored as the last parameters of th)."""
    S = np.outer(th[:p], th[:p]) + np.diag(np.exp(th[p:2 * p]))
    cus = pairs_of(cu)
    for k, pair in enumerate(cus):
        S[pair] += th[len(th) - len(cus) + k]; S[pair[::-1]] += th[len(th) - len(cus) + k]
    return S

def fit(Sxx, Sxz=None, Szz=None, cu=None, start=None):
    """Minimise F = log|S| + tr(S^-1 R), S = ll' + Psi (+ residual covariances cu).
    HA: R = Sxx (age-partialled moments). H0 (Sxz given): item means = l * (c'z), the domain-general restriction."""
    p, q, cus = Sxx.shape[0], (0 if Sxz is None else Sxz.shape[1]), pairs_of(cu)
    def f(th):
        lam, psi, c, S = th[:p], np.exp(th[p:2 * p]), th[2 * p:2 * p + q], sigma(th, cus, p)
        sgn, logdet = np.linalg.slogdet(S)
        if sgn <= 0:
            return 1e10, np.zeros_like(th)
        P, R = np.linalg.inv(S), Sxx
        if q:
            m, v = Sxz @ c, c @ Szz @ c
            R = Sxx - np.outer(m, lam) - np.outer(lam, m) + v * np.outer(lam, lam)
        PR = P @ R
        G = P - PR @ P
        g = np.r_[2 * G @ lam, np.diag(G) * psi, np.zeros(q + len(cus))]
        if q:
            Pl = P @ lam
            g[:p] += -2 * P @ m + 2 * v * Pl
            g[2 * p:2 * p + q] = -2 * Sxz.T @ Pl + 2 * (lam @ Pl) * (Szz @ c)
        for k, pair in enumerate(cus):
            g[2 * p + q + k] = 2 * G[pair]
        return logdet + np.trace(PR), g
    if start is None:
        w, V = np.linalg.eigh(Sxx)
        lam0 = V[:, -1] * np.sqrt(w[-1]) * np.sign(V[:, -1].sum())
        start = np.r_[lam0, np.log(np.clip(np.diag(Sxx) - lam0 ** 2, 0.05, None)), np.zeros(q + len(cus))]
    res = optimize.minimize(f, start, jac=True, method="L-BFGS-B", options={"maxiter": 3000, "ftol": 1e-14, "gtol": 1e-9})
    th = res.x.copy()
    if th[:p].sum() < 0:
        th[:p] *= -1; th[2 * p:2 * p + q] *= -1
    return res.fun, th

def lr_test(Xc, Zc, cu=None, warm=None):
    """Domain-general H0 versus free item-specific age effects HA (same one-factor residual structure); T = N(F0 - F1)."""
    n, p = Xc.shape
    Sxx, Sxz, Szz = Xc.T @ Xc / n, Xc.T @ Zc / n, Zc.T @ Zc / n
    B = np.linalg.solve(Szz, Sxz.T).T
    Sres = Sxx - B @ Sxz.T
    F1, th1 = fit(Sres, cu=cu, start=None if warm is None else warm[0])
    s0 = warm[1] if warm is not None else np.r_[th1[:2 * p], B.T @ th1[:p] / (th1[:p] @ th1[:p]), th1[2 * p:]]
    F0, th0 = fit(Sxx, Sxz, Szz, cu=cu, start=s0)
    return n * (F0 - F1), th0, th1, B, Sres, Sxx, F1

def profile(Xc, z, th1, Sxx):
    """Item-age r; loadings scaled by item SD (proportional to r under H0); LS domain-implied r; deviations; NPS."""
    p = Xc.shape[1]
    sd = np.sqrt(np.diag(Sxx))
    r = (Xc.T @ z[:, 0] / len(Xc)) / (sd * z[:, 0].std())
    ell = th1[:p] / sd
    dev = r - (ell @ r / (ell @ ell)) * ell
    return r, dev, float(dev @ dev / (r @ r))

def boot_null(Xc, Zc, fitted, nb, cu=None, idx=None):
    """Bollen-Stine-type null: residuals rotated to the H0-implied covariance, means regenerated from the H0 fit.
    Resamples are rows of the shared matrix IDX unless another index matrix is given (R8, other sample sizes)."""
    T, th0, th1, B, Sres = fitted[:5]
    p, I = Xc.shape[1], (IDX if idx is None else idx)
    ET = (Xc - Zc @ B.T) @ msqrt(Sres, -0.5) @ msqrt(sigma(th0, cu, p), 0.5)
    M0 = np.outer(Zc @ th0[2 * p:2 * p + Zc.shape[1]], th0[:p])
    Tn, npsn = np.empty(nb), np.empty(nb)
    for b in range(nb):
        Xb, Zb = cen(M0[I[b]] + ET[I[b]]), cen(Zc[I[b]])
        Tn[b], _, t1b, _, _, Sxb, _ = lr_test(Xb, Zb, cu=cu, warm=(th1, th0))
        npsn[b] = profile(Xb, Zb, t1b, Sxb)[2] if Zc.shape[1] == 1 else 0.0
    return (1 + (Tn >= T).sum()) / (nb + 1), Tn, npsn

def rank1(Xc, Zc):  # loading-free test that all items share ONE age curve: Bartlett chi-square, df (p-1)(q-1)
    n, p, q = len(Xc), Xc.shape[1], Zc.shape[1]
    Sxx, Sxz, Szz = Xc.T @ Xc / n, Xc.T @ Zc / n, Zc.T @ Zc / n
    w, V = np.linalg.eigh(msqrt(Sxx, -0.5) @ Sxz @ np.linalg.solve(Szz, Sxz.T) @ msqrt(Sxx, -0.5))
    B = np.linalg.solve(Szz, Sxz.T).T
    stat = -(n - 1 - (p + q + 1) / 2) * np.log(1 - np.clip(w[::-1][1:q], 0, 1 - 1e-12)).sum()
    return stat, msqrt(Sxx, 0.5) @ np.outer(V[:, -1], V[:, -1]) @ msqrt(Sxx, -0.5) @ B, B

test_entry = lambda st, k, pb: {"chi2": R4(st), "df": k, "p_asymptotic": float(stats.chi2.sf(st, k)), "p_bootstrap": R4(pb)}
fisher = lambda r, n: [R4(np.tanh(np.arctanh(r) + s * 1.959964 / np.sqrt(n - 3))) for s in (-1, 1)]
lat = lambda t, ph, p=5: t[2 * p] * np.sqrt(ph) / np.sqrt(t[2 * p] ** 2 * ph + 1)   # latent age correlation under H0
z1 = cen((age / 10)[:, None])                                                  # age in decades, centred
Z3, knots = rcs(age)
IDX = rng.integers(0, N, size=(B_NULL, N))                                     # shared, seeded bootstrap resamples
crit = {a: optimize.brentq(lambda L: stats.ncx2.sf(stats.chi2.ppf(1 - a, 4), 4, L) - 0.80, 0.1, 200) for a in (0.05, 0.01)}

# ------------------------------------------------------------------ H1: per-domain structural (proportionality) tests
out, items_out, DEVB, DEVB_ADJ, FITS = {}, {}, [], [], {}
for dm, name in DOMS.items():
    Xc = cen(X[:, cols[dm]])
    fitted = lr_test(Xc, z1)
    FITS[dm] = fitted
    T, th0, th1, B, Sres, Sxx, F1 = fitted
    r, dev, nps = profile(Xc, z1, th1, Sxx)
    p_boot, Tn, npsn = boot_null(Xc, z1, fitted, B_NULL)
    Wd = np.column_stack([female, EDU, acq(dm)])                               # gender, education, acquiescence
    Xa, za = resid_on(Xc, Wd), resid_on(z1, Wd)
    fa = lr_test(Xa, za)
    dev_adj = profile(Xa, za, fa[2], fa[5])[1]
    devb, deva, npsb, latb = np.empty((B_NP, 5)), np.empty((B_NP, 5)), np.empty(B_NP), np.empty(B_NP)
    for b in range(B_NP):                                                      # nonparametric bootstrap CIs
        Xb, Zb, Xab, Zab = cen(Xc[IDX[b]]), cen(z1[IDX[b]]), cen(Xa[IDX[b]]), cen(za[IDX[b]])
        _, t0b, t1b, _, _, Sxb, _ = lr_test(Xb, Zb, warm=(th1, th0))
        _, devb[b], npsb[b] = profile(Xb, Zb, t1b, Sxb)
        latb[b] = lat(t0b, Zb[:, 0].var())
        _, _, t1a, _, _, Sxa, _ = lr_test(Xab, Zab, warm=(fa[2], fa[1]))
        deva[b] = profile(Xab, Zab, t1a, Sxa)[1]
    DEVB.append(devb); DEVB_ADJ.append(deva)
    S0, sdx, sdz = sigma(th0, None), np.sqrt(np.diag(Sxx)), z1[:, 0].std()
    def ncp(delta, j):  # noncentrality when item j alone has a direct age effect delta (r metric)
        beta = th0[:5] * th0[10] + np.eye(5)[j] * delta * sdx[j] / sdz
        F0p, _ = fit(np.outer(beta, beta) * sdz ** 2 + S0, beta[:, None] * sdz ** 2, np.array([[sdz ** 2]]), start=th0)
        return N * (F0p - np.linalg.slogdet(S0)[1] - 5)
    dl = [np.sqrt(crit[0.01] / (ncp(0.05, j) / 0.0025)) for j in range(5)]
    mde = [np.sqrt(crit[0.01] / (ncp(dl[j], j) / dl[j] ** 2)) for j in range(5)]   # 80% power at alpha = .01
    fs = lr_test(Xc, Z3)                                                        # robustness: spline age
    sres = (Sres - sigma(th1, None)) / np.sqrt(np.outer(np.diag(Sres), np.diag(Sres)))
    srmr = np.sqrt((sres[np.tril_indices(5)] ** 2).mean())
    np.fill_diagonal(sres, 0)
    cu = tuple(sorted(int(i) for i in np.unravel_index(np.abs(sres).argmax(), sres.shape)))
    fc = lr_test(Xc, z1, cu=cu)                                                 # robustness: correlated uniqueness
    Tr, B1, Bo = rank1(Xc, Z3)                                                  # robustness: loading-free rank one
    E3 = Xc - Z3 @ Bo.T
    Trn = np.array([rank1(cen(Z3[IDX[b]] @ B1.T + E3[IDX[b]]), cen(Z3[IDX[b]]))[0] for b in range(B_ROB)])
    chi = N * (F1 - np.linalg.slogdet(Sres)[1] - 5)
    chi0 = N * (np.log(np.diag(Sres)).sum() - np.linalg.slogdet(Sres)[1])
    k5 = X[:, cols[dm]]
    rs, lam1, psi1 = np.corrcoef(k5.mean(1), age)[0, 1], th1[:5], np.exp(th1[5:10])
    out[dm] = {
        "primary": test_entry(T, 4, p_boot), "null_chi2_p95": R4(np.percentile(Tn, 95)),
        "nps": R4(nps), "nps_ci": CI(npsb), "nps_null_mean": R4(npsn.mean()), "nps_null_p95": R4(np.percentile(npsn, 95)),
        "domain_score_age_r": R4(rs), "domain_score_age_r_ci": fisher(rs, N), "latent_age_corr": R4(lat(th0, z1[:, 0].var())),
        "latent_age_corr_ci": CI(latb), "mde_direct_r_median": R4(np.median(mde)), "mde_direct_r_range": [R4(min(mde)), R4(max(mde))],
        "spline": test_entry(fs[0], 12, boot_null(Xc, Z3, fs, B_ROB)[0]), "adjusted": test_entry(fa[0], 4, boot_null(Xa, za, fa, B_ROB)[0]),
        "cu": test_entry(fc[0], 4, boot_null(Xc, z1, fc, B_ROB, cu=cu)[0]), "cu_pair": [ITEMS[cols[dm][cu[0]]], ITEMS[cols[dm][cu[1]]]],
        "cu_resid_corr": R4(sres[cu]), "rank1": test_entry(Tr, 8, (1 + (Trn >= Tr).sum()) / (B_ROB + 1)),
        "fit_chi2_df5": R4(chi), "fit_cfi": R4(1 - max(chi - 5, 0) / max(chi0 - 10, chi - 5, 1e-12)),
        "fit_rmsea": R4(np.sqrt(max(chi - 5, 0) / (5 * (N - 1)))), "fit_srmr": R4(srmr),
        "alpha": R4(1.25 * (1 - k5.var(0, ddof=1).sum() / k5.sum(1).var(ddof=1))), "omega": R4(lam1.sum() ** 2 / (lam1.sum() ** 2 + psi1.sum()))}
    for j, c in enumerate(cols[dm]):
        items_out[ITEMS[c]] = {"r": R4(r[j]), "r_ci": fisher(r[j], N), "loading_std": R4(lam1[j] / np.sqrt(Sres[j, j])),
                               "implied_r": R4(r[j] - dev[j]), "dev": R4(dev[j]), "dev_ci": CI(devb[:, j]),
                               "dev_adj": R4(dev_adj[j]), "dev_adj_ci": CI(deva[:, j])}
    print(f"{name}: chi2(4) = {T:.2f}, boot p = {p_boot:.4f}, NPS = {nps:.3f}; p spline/adjusted/CU/rank-1 = "
          f"{[out[dm][k]['p_bootstrap'] for k in ('spline', 'adjusted', 'cu', 'rank1')]}  [{time.time() - T0:.1f}s]")

def holm(pvals):  # Holm step-down adjusted p values, in input order
    adj, running = {}, 0.0
    for i, k in enumerate(sorted(pvals, key=pvals.get)):
        running = max(running, min(1.0, (len(pvals) - i) * pvals[k]))
        adj[k] = R4(running)
    return adj

running = 0.0  # Holm step-down over the five primary bootstrap p values
for i, dm in enumerate(sorted(DOMS, key=lambda k: out[k]["primary"]["p_bootstrap"])):
    running = max(running, min(1.0, (5 - i) * out[dm]["primary"]["p_bootstrap"]))
    out[dm]["p_holm"] = R4(running)
n_rejected = int(sum(out[dm]["p_holm"] < 0.05 for dm in DOMS))
mixed = [ITEMS.index(i) for i in ITEMS if i[0] != "N"]   # exploratory keying contrast in the four mixed-key domains:
key = np.array([1.0 if ITEMS[i] in REV else 0.0 for i in mixed])   # mean deviation, reverse-keyed minus positively keyed
contrast = lambda D: D[:, mixed] @ (key / key.sum() - (1 - key) / (1 - key).sum())
keying = {lab: {"d": R4(contrast(np.array([[items_out[i][k] for i in ITEMS]]))[0]), "ci": CI(contrast(np.hstack(Db)))}
          for lab, Db, k in (("unadjusted", DEVB, "dev"), ("adjusted", DEVB_ADJ, "dev_adj"))}

# ------------------------------------------------------------------ H2: cross-validated age prediction
def ols_pred(Atr, ytr, Ate):
    coef = np.linalg.lstsq(np.column_stack([np.ones(len(Atr)), Atr]), ytr, rcond=None)[0]
    return np.column_stack([np.ones(len(Ate)), Ate]) @ coef

models = [f"{dm}_{k}" for dm in DOMS for k in ("unit", "factor", "items")] + ["all_unit", "all_factor", "all_items"]
P = {m: np.zeros((REPS, N)) for m in models}
U = np.column_stack([X[:, cols[dm]].mean(1) for dm in DOMS])
warm_fa = {dm: fit(np.cov(X[:, cols[dm]].T, bias=True))[1] for dm in DOMS}
for rep in range(REPS):
    fold = rng.permutation(N) % FOLDS
    for k in range(FOLDS):
        tr, te = fold != k, fold == k
        Ftr, Fte = [], []
        for j, dm in enumerate(DOMS):
            Xd = X[:, cols[dm]]
            mu, (_, thf) = Xd[tr].mean(0), fit(np.cov(Xd[tr].T, bias=True), start=warm_fa[dm])
            w = thf[:5] / np.exp(thf[5:10])                       # Bartlett weights, estimated in the training folds
            Ftr.append((Xd[tr] - mu) @ w); Fte.append((Xd[te] - mu) @ w)
            P[f"{dm}_unit"][rep, te] = ols_pred(U[tr, j:j + 1], age[tr], U[te, j:j + 1])
            P[f"{dm}_factor"][rep, te] = ols_pred(Ftr[-1][:, None], age[tr], Fte[-1][:, None])
            P[f"{dm}_items"][rep, te] = ols_pred(Xd[tr], age[tr], Xd[te])
        P["all_unit"][rep, te] = ols_pred(U[tr], age[tr], U[te])
        P["all_factor"][rep, te] = ols_pred(np.column_stack(Ftr), age[tr], np.column_stack(Fte))
        P["all_items"][rep, te] = ols_pred(X[tr], age[tr], X[te])
Wb = rng.multinomial(N, np.full(N, 1 / N), size=B_CV).astype(float)    # participant bootstrap of out-of-fold errors
sst = Wb @ age ** 2 - (Wb @ age) ** 2 / N
R2b = {m: (1 - (Wb @ ((age[None, :] - P[m]) ** 2).T) / sst[:, None]).mean(1) for m in models}
R2 = {m: float(np.mean(1 - ((age - P[m]) ** 2).sum(1) / ((age - age.mean()) ** 2).sum())) for m in models}
pred = {m: {"r2": R4(R2[m]), "ci": CI(R2b[m])} for m in models}
diff = lambda a, b: {"d": R4(R2[a] - R2[b]), "ci": CI(R2b[a] - R2b[b])}
for dm in list(DOMS) + ["all"]:
    pred[f"{dm}_items_minus_factor"], pred[f"{dm}_factor_minus_unit"] = diff(f"{dm}_items", f"{dm}_factor"), diff(f"{dm}_factor", f"{dm}_unit")
pred["all_items_minus_unit"] = diff("all_items", "all_unit")
for m in ("all_unit", "all_factor"):
    pred[f"share_{m}_of_items"] = {"ratio": R4(R2[m] / R2["all_items"]), "ci": CI(R2b[m] / R2b["all_items"])}
for m in ("all_unit", "all_items"):
    pred[f"{m}_r_pred_obs"] = R4(np.mean([np.corrcoef(P[m][i], age)[0, 1] for i in range(REPS)]))
h2_domains = int(sum(pred[f"{dm}_items_minus_factor"]["ci"][0] > 0 for dm in DOMS))
print("prediction done", f"[{time.time() - T0:.1f}s]", {m: pred[m]["r2"] for m in ("all_unit", "all_factor", "all_items")})

# ------------------------------------------------------------------ H3: split-half replication of item deviations
rep_r, null_r = np.empty(SPLITS), np.empty(SPLITS)
for s in range(SPLITS):
    perm, dv, dp = rng.permutation(N), [], []
    for h in (perm[: N // 2], perm[N // 2:]):
        zh, zp = cen((age[h] / 10)[:, None]), cen((rng.permutation(age[h]) / 10)[:, None])   # zp: age-permutation null
        a_, b_ = [], []
        for dm in DOMS:
            Xh = cen(X[h][:, cols[dm]])
            Sx, Bh = Xh.T @ Xh / len(h), (Xh.T @ zh / len(h)) / zh[:, 0].var()
            _, t1h = fit(Sx - Bh @ Bh.T * zh[:, 0].var(), start=warm_fa[dm][:10])
            a_.append(profile(Xh, zh, t1h, Sx)[1]); b_.append(profile(Xh, zp, t1h, Sx)[1])
        dv.append(np.concatenate(a_)); dp.append(np.concatenate(b_))
    rep_r[s], null_r[s] = np.corrcoef(dv[0], dv[1])[0, 1], np.corrcoef(dp[0], dp[1])[0, 1]
med = float(np.median(rep_r))
replication = {"median_cross_half_r": R4(med), "range_2_5_97_5": CI(rep_r), "null_median": R4(np.median(null_r)),
               "null_p95": R4(np.percentile(null_r, 95)), "spearman_brown_full_sample": R4(2 * med / (1 + med)),
               "share_splits_above_null_p95": R4((rep_r > np.percentile(null_r, 95)).mean()), "splits": SPLITS}
excl0 = lambda k: int(sum(v[k][0] > 0 or v[k][1] < 0 for v in items_out.values()))
print("replication done", f"[{time.time() - T0:.1f}s]")

# ------------------------------------------------------------------ revision analyses (no new random draws)
def mimic_fit(Xc, Zc, th, cu=None, B=None):
    """Fit of a MIMIC model to the joint covariance of the items and age. B=None: H0 (age effects = loadings x c);
    B given: HA (free OLS age slopes). Chi-square, CFI against the independence model of all p + q variables, RMSEA, SRMR."""
    n, p = Xc.shape
    q, k = Zc.shape[1], len(pairs_of(cu))
    D = np.column_stack([Xc, Zc])
    Sj, Szz = D.T @ D / n, Zc.T @ Zc / n
    Bm = np.outer(th[:p], th[2 * p:2 * p + q]) if B is None else B
    Sig = np.block([[Bm @ Szz @ Bm.T + sigma(th, cu, p), Bm @ Szz], [Szz @ Bm.T, Szz]])
    m = p + q
    chi = n * (np.linalg.slogdet(Sig)[1] + np.trace(np.linalg.solve(Sig, Sj)) - np.linalg.slogdet(Sj)[1] - m)
    npar = 2 * p + k + q * (q + 1) // 2 + (q if B is None else p * q)
    dfm = m * (m + 1) // 2 - npar
    chib, dfb = n * (np.log(np.diag(Sj)).sum() - np.linalg.slogdet(Sj)[1]), m * (m - 1) // 2
    res = (Sj - Sig) / np.sqrt(np.outer(np.diag(Sj), np.diag(Sj)))
    return {"chi2": R4(chi), "df": int(dfm), "cfi": R4(1 - max(chi - dfm, 0) / max(chib - dfb, chi - dfm, 1e-12)),
            "rmsea": R4(np.sqrt(max(chi - dfm, 0) / (dfm * (n - 1)))) if dfm > 0 else None,
            "srmr": R4(np.sqrt((res[np.tril_indices(m)] ** 2).mean()))}

def measurement_fit(fitted, cu, p):  # one-factor measurement model with age partialled (HA): chi-square, df, RMSEA
    chi = N * (fitted[6] - np.linalg.slogdet(fitted[4])[1] - p)
    dfm = p * (p + 1) // 2 - 2 * p - len(pairs_of(cu))
    return chi, dfm, (np.sqrt(max(chi - dfm, 0) / (dfm * (N - 1))) if dfm > 0 else 0.0)

def nps_boot(Xc, Zc, fitted, nb, cu=None):  # nonparametric bootstrap of the NPS (rows of IDX)
    th0, th1 = fitted[1], fitted[2]
    vals = np.empty(nb)
    for b in range(nb):
        Xb, Zb = cen(Xc[IDX[b]]), cen(Zc[IDX[b]])
        _, _, t1b, _, _, Sxb, _ = lr_test(Xb, Zb, cu=cu, warm=(th1, th0))
        vals[b] = profile(Xb, Zb, t1b, Sxb)[2]
    return vals

def sensitivity_test(Xc, cu=None):
    """H0-vs-HA test, Bollen-Stine p (B_ROB draws), NPS with nonparametric CI and H0 distribution, MIMIC fit of H0 and HA."""
    p = Xc.shape[1]
    fitted = lr_test(Xc, z1, cu=cu)
    T, th0, th1, B, Sres, Sxx, F1 = fitted
    pb, Tn, npsn = boot_null(Xc, z1, fitted, B_ROB, cu=cu)
    nps = profile(Xc, z1, th1, Sxx)[2]
    return fitted, {"test": test_entry(T, p - 1, pb), "nps": R4(nps), "nps_ci": CI(nps_boot(Xc, z1, fitted, B_ROB, cu=cu)),
                    "nps_null_p95": R4(np.percentile(npsn, 95)),
                    "loading_std": [R4(th1[j] / np.sqrt(Sres[j, j])) for j in range(p)],
                    "mimic_h0": mimic_fit(Xc, z1, th0, cu), "mimic_ha": mimic_fit(Xc, z1, th1, cu, B=B)}

revision = {"mimic_fit_original": {}, "acceptable_fit": {}, "positive_keyed": {}, "eq3": {}, "estimator_check": {}}
# R1: fit of the H0 and HA MIMIC models with the original one-factor structure
for dm in DOMS:
    T, th0, th1, B = FITS[dm][:4]
    revision["mimic_fit_original"][dm] = {"h0": mimic_fit(cen(X[:, cols[dm]]), z1, th0), "ha": mimic_fit(cen(X[:, cols[dm]]), z1, th1, B=B)}

# R2: acceptable-fit measurement models (free residual covariances until RMSEA <= .08), H0 and HA sharing the structure
for dm in DOMS:
    Xc, cus = cen(X[:, cols[dm]]), []
    fitted = FITS[dm]
    chi_m, df_m, rmsea_m = measurement_fit(fitted, cus, 5)
    path = [R4(rmsea_m)]
    while rmsea_m > 0.08 and len(cus) < 3:
        S1 = fitted[4]
        sr = (S1 - sigma(fitted[2], cus, 5)) / np.sqrt(np.outer(np.diag(S1), np.diag(S1)))
        np.fill_diagonal(sr, 0)
        for pair in cus:
            sr[pair] = sr[pair[::-1]] = 0
        cus.append(tuple(sorted(int(i) for i in np.unravel_index(np.abs(sr).argmax(), sr.shape))))
        fitted = lr_test(Xc, z1, cu=cus)
        chi_m, df_m, rmsea_m = measurement_fit(fitted, cus, 5)
        path.append(R4(rmsea_m))
    fitted_s, entry = sensitivity_test(Xc, cu=cus or None)
    th1, S1 = fitted_s[2], fitted_s[4]
    entry.update({"freed_pairs": [[ITEMS[cols[dm][a]], ITEMS[cols[dm][b]]] for a, b in cus],
                  "freed_residual_cov_std": [R4(th1[len(th1) - len(cus) + k] / np.sqrt(S1[a, a] * S1[b, b])) for k, (a, b) in enumerate(cus)],
                  "measurement_chi2": R4(chi_m), "measurement_df": int(df_m), "measurement_rmsea": R4(rmsea_m),
                  "rmsea_path": path, "dev": {ITEMS[c]: R4(v_) for c, v_ in zip(cols[dm], profile(Xc, z1, th1, fitted_s[5])[1])}})
    revision["acceptable_fit"][dm] = entry
    print(f"R2 {DOMS[dm]}: pairs {entry['freed_pairs']}, RMSEA {rmsea_m:.3f}, chi2(4) = {entry['test']['chi2']:.2f}, "
          f"boot p = {entry['test']['p_bootstrap']:.3f}  [{time.time() - T0:.1f}s]")
af_holm = holm({dm: revision["acceptable_fit"][dm]["test"]["p_bootstrap"] for dm in DOMS})
for dm in DOMS:
    revision["acceptable_fit"][dm]["p_holm"] = af_holm[dm]
revision["acceptable_fit_n_rejected_holm"] = int(sum(af_holm[dm] < 0.05 for dm in DOMS))

# R3: positively keyed items only (N has no reverse-keyed items and is unchanged)
for dm in DOMS:
    keep = [c for c in cols[dm] if ITEMS[c] not in REV]
    fitted_s, entry = sensitivity_test(cen(X[:, keep]))
    dev_pos = profile(cen(X[:, keep]), z1, fitted_s[2], fitted_s[5])[1]
    entry.update({"items": [ITEMS[c] for c in keep], "dev": {ITEMS[c]: R4(dev_pos[j]) for j, c in enumerate(keep)}})
    revision["positive_keyed"][dm] = entry
    print(f"R3 {DOMS[dm]}: {entry['items']}, chi2({entry['test']['df']}) = {entry['test']['chi2']:.2f}, "
          f"boot p = {entry['test']['p_bootstrap']:.3f}  [{time.time() - T0:.1f}s]")
pk_holm = holm({dm: revision["positive_keyed"][dm]["test"]["p_bootstrap"] for dm in DOMS})
for dm in DOMS:
    revision["positive_keyed"][dm]["p_holm"] = pk_holm[dm]
revision["positive_keyed_n_rejected_holm"] = int(sum(pk_holm[dm] < 0.05 for dm in DOMS))

# R4: moments behind the loading-drift bound (Eq. 3)
za_ = z1[:, 0]
sa, mu3 = za_.std(), np.mean(za_ ** 3)
skew = mu3 / sa ** 3
revision["eq3"] = {"age_sd_decades": R4(sa), "age_mu3_decades3": R4(mu3), "age_skewness": R4(skew), "domains": {}}
for dm in DOMS:
    g = FITS[dm][1][10]
    revision["eq3"]["domains"][dm] = {"gamma_per_decade": R4(g), "gamma_x_sd_age": R4(g * sa),
                                      "loading_drift_needed_d04": R4(0.04 / abs(g * sa * skew)),
                                      "loading_drift_needed_d08": R4(0.08 / abs(g * sa * skew))}

# R5: estimator cross-check
def joint_ml(D, h0):
    """Independent ML fit of the joint (items, age) MIMIC covariance: different parameterisation, numerical gradients."""
    n, m = D.shape
    p = m - 1
    S = D.T @ D / n
    def implied(t):
        lam, psi, va = t[:p], np.exp(t[p:2 * p]), np.exp(t[-1])
        Bv = lam * t[2 * p] if h0 else t[2 * p:3 * p]
        return np.block([[np.outer(Bv, Bv) * va + np.outer(lam, lam) + np.diag(psi), (Bv * va)[:, None]],
                         [(Bv * va)[None, :], np.array([[va]])]])
    def F(t):
        Sg = implied(t)
        sgn, ld = np.linalg.slogdet(Sg)
        return ld + np.trace(np.linalg.solve(Sg, S)) if sgn > 0 else 1e10
    t0 = np.r_[0.7 * np.sqrt(np.diag(S)[:p]), np.log(0.5 * np.diag(S)[:p]), np.zeros(1 if h0 else p), np.log(S[p, p])]
    res = optimize.minimize(F, t0, method="BFGS", options={"gtol": 1e-9, "maxiter": 20000})
    t = res.x.copy()
    if t[:p].sum() < 0:
        t[:p] *= -1
        if h0:
            t[2 * p] *= -1
    return n * (res.fun - np.linalg.slogdet(S)[1] - m), t

dl_sk, dchi_sk, dT_ind, dchi_ind, dl_ind = [], [], [], [], []
for dm in DOMS:
    Xc = cen(X[:, cols[dm]])
    T, th0, th1, B, Sres, Sxx, F1 = FITS[dm]
    fa_sk = FactorAnalysis(n_components=1, tol=1e-12, max_iter=100000, svd_method="lapack").fit(Xc - z1 @ B.T)
    S_sk, l_sk = fa_sk.get_covariance(), fa_sk.components_[0] * np.sign(fa_sk.components_[0].sum())
    chi_sk = N * (np.linalg.slogdet(S_sk)[1] + np.trace(np.linalg.solve(S_sk, Sres)) - np.linalg.slogdet(Sres)[1] - 5)
    sdr = np.sqrt(np.diag(Sres))
    chi1 = N * (F1 - np.linalg.slogdet(Sres)[1] - 5)
    dl_sk.append(np.abs(l_sk / sdr - th1[:5] / sdr).max()); dchi_sk.append(abs(chi_sk - chi1))
    D = np.column_stack([Xc, z1])
    c0, t0_ = joint_ml(D, True)
    c1, t1_ = joint_ml(D, False)
    dT_ind.append(abs((c0 - c1) - T)); dchi_ind.append(abs(c1 - chi1))
    dl_ind.append(max(np.abs(t1_[:5] / sdr - th1[:5] / sdr).max(), np.abs(t0_[:5] / sdr - th0[:5] / sdr).max()))
    revision["estimator_check"][dm] = {"chi2_df4_custom": R4(T), "chi2_df4_independent": R4(c0 - c1),
                                       "chi2_df5_custom": R4(chi1), "chi2_df5_sklearn": R4(chi_sk), "chi2_df5_independent": R4(c1)}
revision["estimator_check"]["max_abs_diff"] = {"loading_std_sklearn": float(max(dl_sk)), "chi2_df5_sklearn": float(max(dchi_sk)),
                                               "chi2_df4_independent": float(max(dT_ind)), "chi2_df5_independent": float(max(dchi_ind)),
                                               "loading_std_independent": float(max(dl_ind))}
print("revision analyses done", f"[{time.time() - T0:.1f}s]", revision["estimator_check"]["max_abs_diff"])

# ------------------------------------------------------------------ revision analyses R6-R8 (only R8b draws, from its own stream)
def meas_fit(Sres, th1, cu, p=5, n=N):
    """Age-partialled one-factor measurement model: chi-square, df, CFI (independence baseline), RMSEA, SRMR."""
    S = sigma(th1, cu, p)
    chi = n * (np.linalg.slogdet(S)[1] + np.trace(np.linalg.solve(S, Sres)) - np.linalg.slogdet(Sres)[1] - p)
    dfm = p * (p + 1) // 2 - 2 * p - len(pairs_of(cu))
    chib, dfb = n * (np.log(np.diag(Sres)).sum() - np.linalg.slogdet(Sres)[1]), p * (p - 1) // 2
    sr = (Sres - S) / np.sqrt(np.outer(np.diag(Sres), np.diag(Sres)))
    return {"chi2": R4(chi), "df": int(dfm), "cfi": R4(1 - max(chi - dfm, 0) / max(chib - dfb, chi - dfm, 1e-12)),
            "rmsea": R4(np.sqrt(max(chi - dfm, 0) / (dfm * (n - 1)))), "srmr": R4(np.sqrt((sr[np.tril_indices(p)] ** 2).mean()))}

# R6: fit of the robustness models' measurement structures (one correlated uniqueness; covariate/acquiescence-adjusted)
revision["cu_fit"], revision["adjusted_fit"], DEV0, DEVA0 = {}, {}, [], []
for dm in DOMS:
    Xc = cen(X[:, cols[dm]])
    T, th0, th1, B, Sres, Sxx, F1 = FITS[dm]
    DEV0.append(profile(Xc, z1, th1, Sxx)[1])
    pair = tuple(sorted(ITEMS.index(i) - cols[dm][0] for i in out[dm]["cu_pair"]))
    fc = lr_test(Xc, z1, cu=pair)
    revision["cu_fit"][dm] = {"pair": out[dm]["cu_pair"], "test_chi2": R4(fc[0]), "measurement": meas_fit(fc[4], fc[2], pair),
                              "mimic_h0": mimic_fit(Xc, z1, fc[1], pair), "mimic_ha": mimic_fit(Xc, z1, fc[2], pair, B=fc[3])}
    Wd = np.column_stack([female, EDU, acq(dm)])
    Xa, za = resid_on(Xc, Wd), resid_on(z1, Wd)
    fa = lr_test(Xa, za)
    DEVA0.append(profile(Xa, za, fa[2], fa[5])[1])
    revision["adjusted_fit"][dm] = dict(meas_fit(fa[4], fa[2], None), test_chi2=R4(fa[0]))
revision["fit_reproduces_table1"] = bool(all(meas_fit(FITS[dm][4], FITS[dm][2], None)["chi2"] == out[dm]["fit_chi2_df5"] for dm in DOMS))

# R7: multiplicity-adjusted inference for the 25 item deviations from the same 999 nonparametric resamples:
# simultaneous 95% band (max-|t| over the 25 items, t studentised by the bootstrap SD), single-step max-T adjusted p,
# and Holm-adjusted p values from the bootstrap z = deviation / bootstrap SD
def holm_raw(p):
    p, adj, running = np.asarray(p, float), np.empty(len(p)), 0.0
    for i, k in enumerate(np.argsort(p, kind="stable")):
        running = max(running, min(1.0, (len(p) - i) * p[k]))
        adj[k] = running
    return adj

def sim_band(Db, est):
    D = np.hstack(Db)
    se = D.std(0, ddof=1)
    tmax = (np.abs(D - est) / se).max(1)
    c = float(np.percentile(tmax, 95))
    lo, hi = est - c * se, est + c * se
    p_maxt = np.array([(1 + (tmax >= abs(e) / s).sum()) / (len(tmax) + 1) for e, s in zip(est, se)])
    p_holm = holm_raw(2 * stats.norm.sf(np.abs(est / se)))
    inside = float(((D >= lo) & (D <= hi)).all(1).mean())
    items = {ITEMS[j]: {"dev": R4(est[j]), "boot_sd": R4(se[j]), "band": [R4(lo[j]), R4(hi[j])], "p_maxT": R4(p_maxt[j]),
                        "p_holm": float(p_holm[j])} for j in range(len(ITEMS))}
    ts, k = np.sort(tmax), 1.959964 * np.sqrt(len(tmax) * 0.95 * 0.05)          # Monte Carlo 95% interval of c (order statistics)
    c_mc = [R4(ts[int(np.floor(len(tmax) * 0.95 - k)) - 1]), R4(ts[min(int(np.ceil(len(tmax) * 0.95 + k)) - 1, len(tmax) - 1)])]
    return {"critical_value": R4(c), "critical_value_mc_95": c_mc, "share_resamples_inside_band": R4(inside), "items": items,
            "n_band_excludes_0": int(((lo > 0) | (hi < 0)).sum()), "n_holm_below_05": int((p_holm < 0.05).sum()),
            "band_excludes_0": [ITEMS[j] for j in range(len(ITEMS)) if lo[j] > 0 or hi[j] < 0],
            "holm_below_05": [ITEMS[j] for j in range(len(ITEMS)) if p_holm[j] < 0.05]}

revision["item_bands"] = {"unadjusted": sim_band(DEVB, np.concatenate(DEV0)), "adjusted": sim_band(DEVB_ADJ, np.concatenate(DEVA0))}
print("R7 simultaneous bands: items excluding 0 =", revision["item_bands"]["unadjusted"]["band_excludes_0"],
      "| Holm =", revision["item_bands"]["unadjusted"]["holm_below_05"], f"[{time.time() - T0:.1f}s]")

# R8: missing-data sensitivity - full-information ML (FIML) of the primary H0-vs-HA test on ALL age-eligible participants
# (18-65, n = 2,543), conditional on age (complete for everyone), so MAR given age and the observed items is assumed.
# Patterns of missing items are grouped and the likelihood is evaluated from per-pattern sufficient statistics.
def fiml_prep(Xm, a):
    obs = ~np.isnan(Xm)
    use = obs.any(1)
    Xm, a, obs = Xm[use], a[use], obs[use]
    pats = []
    for key in np.unique(obs, axis=0):
        rows, o = (obs == key).all(1), np.flatnonzero(key)
        x, aa = Xm[rows][:, o], a[rows]
        pats.append((o, int(rows.sum()), x.sum(0), x.T @ x, aa.sum(), aa @ aa, x.T @ aa))
    return pats, int(use.sum()), int((~obs.all(1)).sum())

def fiml_obj(th, pats, ntot, h0, cu):
    """-2 log L / n of x | age ~ N(nu + B age, ll' + Psi (+ cu)); H0: B = l * gamma. Params: l, log psi, nu, (gamma | B), cu."""
    p = 5
    lam, psi, nu = th[:p], np.exp(th[p:2 * p]), th[2 * p:3 * p]
    Bv = lam * th[3 * p] if h0 else th[3 * p:4 * p]
    S = sigma(th, cu, p)
    G, gnu, gB, f = np.zeros((p, p)), np.zeros(p), np.zeros(p), 0.0
    for o, n, sx, sxx, sa, saa, sxa in pats:
        So = S[np.ix_(o, o)]
        sgn, ld = np.linalg.slogdet(So)
        if sgn <= 0:
            return 1e10, np.zeros_like(th)
        Po, nu_o, B_o = np.linalg.inv(So), nu[o], Bv[o]
        M = (sxx - np.outer(sx, nu_o) - np.outer(nu_o, sx) - np.outer(sxa, B_o) - np.outer(B_o, sxa) + n * np.outer(nu_o, nu_o)
             + sa * (np.outer(nu_o, B_o) + np.outer(B_o, nu_o)) + saa * np.outer(B_o, B_o))
        PM = Po @ M
        f += n * ld + np.trace(PM)
        G[np.ix_(o, o)] += n * Po - PM @ Po
        gnu[o] += -2 * Po @ (sx - n * nu_o - sa * B_o)
        gB[o] += -2 * Po @ (sxa - sa * nu_o - saa * B_o)
    g = np.concatenate([2 * G @ lam + (th[3 * p] * gB if h0 else 0), np.diag(G) * psi, gnu,
                        [lam @ gB] if h0 else gB, [2 * G[pr] for pr in pairs_of(cu)]])
    return f / ntot, g / ntot

def fiml_test(Xm, a, dm):
    """FIML likelihood-ratio statistic of the domain-general restriction, started from the complete-case estimates."""
    pats, ntot, ninc = fiml_prep(Xm, a)
    T, th0, th1, B = FITS[dm][:4]
    nu0 = np.nanmean(Xm, 0) - B[:, 0] * np.nanmean(a)
    opts = {"maxiter": 5000, "ftol": 1e-15, "gtol": 1e-10}
    r1 = optimize.minimize(fiml_obj, np.r_[th1[:10], nu0, B[:, 0]], args=(pats, ntot, False, None), jac=True, method="L-BFGS-B", options=opts)
    r0 = optimize.minimize(fiml_obj, np.r_[th0[:10], nu0, th0[10]], args=(pats, ntot, True, None), jac=True, method="L-BFGS-B", options=opts)
    return ntot * (r0.fun - r1.fun), ntot, ninc, len(pats), r0, r1

elig = df[in_age].reset_index(drop=True)
Xall = elig[ITEMS].to_numpy(float)
for it in REV:
    Xall[:, ITEMS.index(it)] = 7 - Xall[:, ITEMS.index(it)]
a_all = elig.age.to_numpy(float) / 10
a_all = a_all - a_all.mean()
fiml, repro = {"n_age_eligible": int(len(elig)), "n_incomplete_any_item": int(np.isnan(Xall).any(1).sum()), "domains": {}}, []
for dm in DOMS:
    Tcc = fiml_test(X[:, cols[dm]], z1[:, 0], dm)[0]                       # check: FIML on the complete cases = primary T
    repro.append(abs(Tcc - FITS[dm][0]))
    Tf, ntot, ninc, npat, r0, r1 = fiml_test(Xall[:, cols[dm]], a_all, dm)
    fiml["domains"][dm] = {"n_used": ntot, "n_with_missing_items_used": ninc, "n_patterns": npat, "chi2": R4(Tf), "df": 4,
                           "p_asymptotic": float(stats.chi2.sf(Tf, 4)), "converged": bool(r0.success and r1.success),
                           "complete_case_chi2": out[dm]["primary"]["chi2"], "complete_case_p_asymptotic": out[dm]["primary"]["p_asymptotic"]}
fh = holm_raw([fiml["domains"][dm]["p_asymptotic"] for dm in DOMS])
ch = holm_raw([out[dm]["primary"]["p_asymptotic"] for dm in DOMS])
for k, dm in enumerate(DOMS):
    fiml["domains"][dm]["p_holm_asymptotic"], fiml["domains"][dm]["complete_case_p_holm_asymptotic"] = float(fh[k]), float(ch[k])
fiml.update({"n_rejected_holm": int((fh < 0.05).sum()), "complete_case_n_rejected_holm_asymptotic": int((ch < 0.05).sum()),
             "rejected": [dm for k, dm in enumerate(DOMS) if fh[k] < 0.05], "complete_case_reproduction_max_abs_diff": float(max(repro))})
# R8b: available cases per domain (all age-eligible participants complete on the tested domain's five items) with the
# primary analysis's ML test and Bollen-Stine bootstrap (B_ROB draws); its resamples come from a separate seeded stream
# (SeedSequence(20261024).spawn(1)[0]), so no number above changes
rng_md, avail = np.random.default_rng(np.random.SeedSequence(20261024).spawn(1)[0]), {}
for dm in DOMS:
    ok = ~np.isnan(Xall[:, cols[dm]]).any(1)
    Xc_, zc_, n_ = cen(Xall[ok][:, cols[dm]]), cen(a_all[ok][:, None]), int(ok.sum())
    fa_ = lr_test(Xc_, zc_)
    avail[dm] = dict(test_entry(fa_[0], 4, boot_null(Xc_, zc_, fa_, B_ROB, idx=rng_md.integers(0, n_, size=(B_ROB, n_)))[0]), n=n_)
    avail[dm]["p_bootstrap_mc_se"] = R4(np.sqrt(avail[dm]["p_bootstrap"] * (1 - avail[dm]["p_bootstrap"]) / B_ROB))
ah = holm_raw([avail[dm]["p_bootstrap"] for dm in DOMS])
for k, dm in enumerate(DOMS):
    avail[dm]["p_holm"] = R4(ah[k])
fiml["available_cases"] = {"domains": avail, "n_rejected_holm": int((ah < 0.05).sum()), "rejected": [dm for k, dm in enumerate(DOMS) if ah[k] < 0.05]}
revision["fiml"] = fiml
print("R8b available cases:", {dm: (avail[dm]["n"], avail[dm]["chi2"], avail[dm]["p_bootstrap"]) for dm in DOMS},
      "| rejected (Holm):", fiml["available_cases"]["rejected"], f"[{time.time() - T0:.1f}s]")
print("R8 FIML:", {dm: (fiml["domains"][dm]["n_used"], fiml["domains"][dm]["chi2"], round(fiml["domains"][dm]["p_asymptotic"], 5)) for dm in DOMS},
      "| rejected (Holm):", fiml["rejected"], "| reproduction diff:", fiml["complete_case_reproduction_max_abs_diff"], f"[{time.time() - T0:.1f}s]")

res = {"flow": flow, "sample": sample, "spline_knots_years": [R4(t) for t in knots], "domains": out, "items": items_out,
       "keying_contrast": keying, "prediction": pred, "replication": replication,
       "power": {"ncp_80pct_alpha05_df4": R4(crit[0.05]), "ncp_80pct_alpha01_df4": R4(crit[0.01]),
                 "min_detectable_r_alpha05_power80": R4(np.tanh((1.959964 + 0.841621) / np.sqrt(N - 3)))},
       "decision": {"n_domains_rejected_holm": n_rejected, "n_domains_items_beat_factor_ci": h2_domains,
                    "replication_exceeds_null_p95": bool(med > replication["null_p95"]),
                    "n_items_dev_ci_excludes_0": excl0("dev_ci"), "n_items_dev_adj_ci_excludes_0": excl0("dev_adj_ci")},
       "bootstrap": {"B_null": B_NULL, "B_ci": B_NP, "B_robustness": B_ROB, "B_prediction": B_CV, "cv": f"{FOLDS}-fold x {REPS}", "seed": 20261024},
       "revision": revision,
       "software": {"python": platform.python_version(), "numpy": np.__version__, "scipy": scipy.__version__,
                    "pandas": pd.__version__, "scikit_learn": sklearn.__version__},
       "hypothesis_supported": bool(n_rejected >= 4), "runtime_seconds": R4(time.time() - T0)}
print("RESULT_JSON: " + json.dumps(res, separators=(",", ":")))
Listing. experiment_v1.py

4.Analysis

Attempt 4Done

Analysis complete: experiment succeeded; hypothesis supported; 24 key numbers.

Reviewer feedback applied: Independent referee (Claude Fable 5.1), round 2: accept. The editor asks for the referee's two optional proof-stage additions: (1) one sentence in the Supplemental Method reporting the loading-drift bound the archived script computes (R4: drift of the SD-scaled loading per SD of age needed for a deviation of .04 or .08, about .27-.54 for .04); (2) one clause noting that the 25-item complete-case filter served the whole-inventory analyses, whereas each domain's test needs only that domain's items (273 participants with complete openness items were dropped; with them openness is rejected, p = .024 available cases / .014 FIML). Keep the analysis record consistent with the paper; no other change.
Hypothesis supportedExperiment succeeded

Results and analysis

Run status. The extended analysis script (the original analysis plus revision analyses R1-R8 requested in review) ran once without error (exit code 0, 49 s) and printed a valid RESULT_JSON; every number from the first run was reproduced exactly. Sample: 2,209 complete cases aged 18-65 of 2,800 SAPA participants (248 under 18, 9 over 65 and 334 with missing items excluded); the missing-data analyses used all 2,543 age-eligible participants.

H1 (primary) is supported by its prespecified rule (rejection in at least four domains). The domain-general restriction was rejected for agreeableness, conscientiousness, extraversion and neuroticism, chi-square(4) = 17.32-53.59, bootstrap p <= .004, Holm p <= .007, but not for openness, chi-square(4) = 8.37, p = .073. The same four were rejected under measurement models that fit acceptably (conscientiousness with C1-C2 freed: RMSEA = .075, chi-square(4) = 22.09, p = .004; neuroticism with the three N3-N5 residual covariances freed: RMSEA = .000, chi-square(4) = 22.83, p = .002) and with positively keyed items only. Outside neuroticism the rejections rest on the estimated loadings: the loading-free test of a shared age-curve shape rejected only for neuroticism (p = .004). With all age-eligible participants (full-information ML) openness was also rejected, chi-square(4) = 12.55, p = .014, as it was when the available cases restored the participants with complete openness items whom the whole-inventory complete-case filter had dropped (bootstrap p = .024); openness is undetermined rather than domain-general.

Items: 13 of 25 deviations had unadjusted 95% CIs excluding zero, but only 8 survived a simultaneous 95% band and Holm correction (A1, A3, C2, C4, E2, E3, E4, E5); no neuroticism or openness item did.

H2 is not supported by its prespecified rule: items beat the factor score in only 2 of 5 domains, and the share of the items' out-of-sample R-squared recovered by the five domain scores, .39 [.18, .58], does not establish 'less than half'. Only the aggregate contrast favours items (R-squared .067 [.043, .089] vs .026 [.010, .040]).

H3 is supported: item deviations correlated at a median r = .78 across random halves against a null 95th percentile of .37.

Key numbers

MetricValueInterpretation
Sample2,209 of 2,800 analysed (excluded: 248 age < 18, 9 age > 65, 334 with missing items); 2,543 age-eligiblecomplete cases for the primary analysis; all age-eligible participants in the missing-data analyses
Age and genderM = 29.74, SD = 10.39, median 27; 253 aged 45-65; 67.04% women; excluded incomplete cases M = 31.20young, self-selected internet sample; incomplete cases were older
One-factor fit (age partialled)A chi2(5) = 75.44, CFI = .966, RMSEA = .080; C chi2(5) = 127.43, CFI = .942, RMSEA = .105; E chi2(5) = 63.62, CFI = .977, RMSEA = .073; N chi2(5) = 314.68, CFI = .921, RMSEA = .168; O chi2(5) = 67.62, CFI = .947, RMSEA = .075acceptable by RMSEA <= .08 for A, E and O; poor for C and N by the cited cutoffs
Primary restriction tests (H1)A chi2(4) = 45.15, bootstrap p = .0005, Holm p = .003; C chi2(4) = 23.82, bootstrap p = .0005, Holm p = .003; E chi2(4) = 53.59, bootstrap p = .0005, Holm p = .003; N chi2(4) = 17.32, bootstrap p = .0035, Holm p = .007; O chi2(4) = 8.37, bootstrap p = .0725, Holm p = .073rejected in 4 of 5 domains (not openness): H1 supported by the prespecified rule
Acceptable-fit measurement structuresC (C1-C2 freed): RMSEA .075, chi2(4) = 22.09, p = .004; N (N4-N5, N3-N4, N3-N5 freed): RMSEA .000, chi2(4) = 22.83, p = .002; A, E, O unchanged (RMSEA <= .08); Holm: 4 of 5 rejected (O p = .070)the C and N rejections do not depend on the poorly fitting one-factor model
Fit with one correlated uniqueness (measurement model)A A1-A2: chi2(4) = 16.21, CFI = .994, RMSEA = .037, SRMR = .014; C C1-C2: chi2(4) = 53.33, CFI = .977, RMSEA = .075, SRMR = .028; E E3-E5: chi2(4) = 15.77, CFI = .995, RMSEA = .037, SRMR = .012; N N4-N5: chi2(4) = 234.62, CFI = .941, RMSEA = .162, SRMR = .051; O O2-O5: chi2(4) = 21.67, CFI = .985, RMSEA = .045, SRMR = .019one freed pair restores acceptable fit except in neuroticism
Loading-free shared-age-curve testA chi2(8) = 10.98, p = .210; C chi2(8) = 9.48, p = .340; E chi2(8) = 6.95, p = .560; N chi2(8) = 22.67, p = .004; O chi2(8) = 9.60, p = .208rejects only for neuroticism; the A, C and E rejections rest on the estimated loadings and their invariance across age
Positively keyed items onlyA chi2(3) = 25.22, p = .002; C chi2(2) = 13.28, p = .006; E chi2(2) = 31.32, p = .002; N chi2(4) = 17.32, p = .004; O chi2(2) = 1.47, p = .456; Holm: 4 of 5 rejectedthe rejections are not created by reverse keying
Other robustness tests (499 draws)spline chi2(12) p: A .002, C .014, E .002, N .002, O .170; adjusted chi2(4) p: A .002, C .008, E .002, N .006, O .036; correlated uniqueness chi2(4) p: A .002, C .004, E .002, N .002, O .046the four rejections survive flexible age, covariate/acquiescence adjustment and a freed residual pair; openness p = .036 to .170
Missing data: FIML on all age-eligible participantsA chi2(4) = 54.33, p < .001; C chi2(4) = 23.69, p < .001; E chi2(4) = 67.67, p < .001; N chi2(4) = 17.41, p = .002; O chi2(4) = 12.55, p = .014; Holm: 5 of 5 rejected (complete cases, asymptotic: 4 of 5)the four rejections are unchanged; openness is also rejected when the participants dropped by the 25-item complete-case filter are included
Missing data: available cases per domainA n = 2,462, chi2(4) = 53.99, bootstrap p = .002; C n = 2,461, chi2(4) = 24.07, bootstrap p = .002; E n = 2,464, chi2(4) = 65.57, bootstrap p = .002; N n = 2,448, chi2(4) = 16.41, bootstrap p = .004; O n = 2,482, chi2(4) = 11.72, bootstrap p = .024; Monte Carlo SE of O's p = .0069each domain's primary test needs only that domain's items; restoring the participants with complete openness items whom the 25-item filter had dropped, this resampling-calibrated check agrees with FIML, including for openness
Non-proportional share (NPS)A .23 [.13, .44] (null 95th pct. .09); C .26 [.10, .65] (null 95th pct. .19); E .57 [.28, .92] (null 95th pct. .34); N .16 [.06, .45] (null 95th pct. .12); O .28 [.09, .84] (null 95th pct. .52)beyond the null in A, C, E and N but imprecise; openness inside its null distribution
Item deviations, multiplicity13 of 25 unadjusted CIs exclude 0; simultaneous 95% band (c = 3.05, Monte Carlo 95% interval 2.99-3.12) excludes 0 for 8: A1 .084 [.028, .140], A3 -.070 [-.108, -.031], C2 -.063 [-.114, -.012], C4 .047 [.007, .088], E2 .055 [.015, .096], E3 -.065 [-.111, -.018], E4 -.056 [-.101, -.012], E5 .060 [.007, .113]; Holm: 8item-level statements are confined to these 8 items; no neuroticism or openness item survives
Adjusted item deviations, multiplicity13 of 25 unadjusted CIs exclude 0; simultaneous band excludes 0 for 6 (A3, A4, C2, E2, E3, E5)after gender, education and acquiescence adjustment A1 and C4 drop out and A4 enters
Domain score-age rA .156 [.115, .196]; C .087 [.045, .128]; E .067 [.026, .109]; N -.098 [-.139, -.056]; O .057 [.015, .098]small domain-level age differences
Out-of-sample R2, all five domainsitems .067 [.043, .089]; unit scores .026 [.010, .040]; factor scores .019 [.005, .031]; items - unit .041 [.022, .060]; items - factor .048 [.028, .067]only this aggregate contrast favours items
Share of the items' R2 recoveredunit scores .389 [.183, .582]; factor scores .280 [.080, .458]interval includes one half: 'less than half' not established
Items minus factor score within domainsA .016 [.005, .027]; C .007 [-.002, .016]; E .020 [.007, .033]; N .006 [-.002, .013]; O -.001 [-.006, .004]CI excludes 0 in 2 of 5 domains: the prespecified per-domain criterion fails, H2 not supported
Factor minus unit score, all domains-.007 [-.011, -.003]optimal weighting did not recover the age information
Split-half replication (H3)median r = .78 (.65-.89); null 95th pct. .37; 100% of splits above; Spearman-Brown .87H3 supported by its prespecified rule
Keying contrast (exploratory).047 [.029, .067]; adjusted .020 [.007, .035]reverse-keyed items deviate upward; acquiescence adjustment removes part
Sensitivitynoncentrality for 80% power 16.75 (alpha .01), 11.94 (alpha .05); detectable single-item direct effect (median) A .077, C .079, E .076, N .067, O .084; item-age r detectable at alpha .05: .060the openness test is the least sensitive
Estimator cross-checkmax |difference| in standardized loadings 2.7e-07 (sklearn) and 2.8e-07 (independent joint fit); in chi-square <= 3.9e-10; FIML reproduces the complete-case statistic to 3.9e-10estimates do not depend on the custom optimiser
Softwarepython 3.12.13, numpy 2.5.2, scipy 1.18.1, pandas 3.0.5, scikit_learn 1.9.0versions recorded by the executed run

Caveats

  • Cross-sectional data: age is confounded with birth cohort; the results describe age differences, not within-person change.
  • A 2010 demonstration subset (2,800 participants) of the self-selected SAPA internet sample: 67% women and young (median 27; 253 aged 45-65). The 25 items are a demonstration selection of IPIP items, not a validated short Big Five inventory.
  • Single-occasion self-reports: item-specific age differences may reflect age differences in item interpretation, social desirability or response style.
  • Loading-based tests depend on the measurement model. The acceptable-fit structures were found by data-driven respecification (residual covariances freed in order of standardized residual), and the neuroticism structure needed all three N3-N5 covariances, so facet-level and item-level specificity cannot be separated; outside neuroticism the loading-free test, which is sensitive only to differences in the shape of age curves, did not reject. A first-order loading-drift bound computed by the script (R4) indicates that loadings drifting linearly with age, with invariant intercepts, would have to change by amounts large relative to the loadings themselves to produce deviations of the observed size; it is an approximation, not a substitute for moderated factor analysis.
  • The acquiescence index is built from raw responses to other domains' items, which carry trait and age variance of their own, so the adjusted tests remove substantive as well as stylistic variance; the positively-keyed-only tests avoid this but cannot remove acquiescence that raises all positively keyed items.
  • Only 8 of the 25 item deviations survive multiplicity adjustment; the other item-level differences are exploratory.
  • Openness is undetermined: not rejected in the primary complete-case test (p = .073) or with positively keyed items (p = .456), but rejected when the participants with complete openness items whom the 25-item complete-case filter had dropped were included (FIML p = .014; available cases p = .024); that filter served the whole-inventory analyses, whereas each domain's primary test needs only that domain's items.
  • FIML assumes data missing at random given age and the observed items, and its p values are asymptotic (in the complete cases the bootstrap null 95th percentiles, 9.25-10.12, were close to the nominal chi-square(4) reference); the available-case bootstrap uses 499 draws.
  • Six-point items were treated as continuous with normal-theory ML; inference relies on bootstrap calibration rather than categorical estimators.
  • Prediction intervals resample participants given the fold models; per-domain predictive gains are small (Delta R2 <= .02).
  • The script was developed on these data, so the confirmatory run was not blind to them; the revision analyses were added after peer review, and the keying contrast is exploratory.
  • Single dataset with no external replication; larger public SAPA releases exist but were not analysed.

5.Paper writing

Attempt 8Done

Manuscript drafted: “Four Big Five Domains Do Not Carry All of Their Items' Age Differences: A Loading-Based Test in 25 IPIP Items” (≈ 8197 words).

Reviewer feedback applied: Independent referee (Claude Fable 5.1), round 2: accept. The editor asks for the referee's two optional proof-stage additions: (1) one sentence in the Supplemental Method reporting the loading-drift bound the archived script computes (R4: drift of the SD-scaled loading per SD of age needed for a deviation of .04 or .08, about .27-.54 for .04); (2) one clause noting that the 25-item complete-case filter served the whole-inventory analyses, whereas each domain's test needs only that domain's items (273 participants with complete openness items were dropped; with them openness is rejected, p = .024 available cases / .014 FIML). Keep the analysis record consistent with the paper; no other change.

≈ 8197 words · AI-drafted manuscript · not peer-reviewed

Four Big Five Domains Do Not Carry All of Their Items' Age Differences: A Loading-Based Test in 25 IPIP Items

Abstract

Do age differences in personality belong to Big Five domains or to the characteristics their items describe? If age acted on items only through their domain, item–age associations would be proportional to factor loadings. We tested this restriction with multiple-indicators multiple-causes models in 2,209 adults (18–65 years) who answered 25 International Personality Item Pool items. It was rejected for agreeableness, conscientiousness, extraversion and neuroticism, χ²(4) = 17.32 to 53.59, bootstrap ps ≤ .004, also under acceptably fitting measurement models; only for neuroticism did a loading-free test also reject, so the other three rest on the estimated loadings. Openness was undetermined (p = .073; .014 to .456 in other specifications). Departures replicated across halves (median r = .78). Out of sample, the 25 items predicted age better than the five domain scores, R² = .067, 95% CI [.043, .089], versus .026 [.010, .040], but beat factor scores within only two domains.

Keywords: personality development; age differences; Big Five; personality nuances; measurement invariance; MIMIC models; cross-validation

Statement of Relevance

On average, adults become more agreeable, conscientious and emotionally stable as they age, differences usually described with five broad traits. If a trait changed as a single unit, every questionnaire item would differ with age in proportion to how well it measures the trait. In 2,209 adults this failed for four of the five traits; for openness the evidence was mixed. Some item-level differences survived correction for testing 25 items, and the pattern of item differences replicated across halves of the sample. Older adults, for example, described themselves as more likely to take charge, but not as more captivating or quicker to make friends. Across all five traits the items predicted age better than the trait scores, although within single traits they beat optimally weighted trait scores reliably for only two. Researchers who compare age groups on trait scores can check whether the items tell the same story.

Introduction

Across adulthood, people on average become more agreeable, conscientious and emotionally stable (Bleidorn et al., 2022; Roberts et al., 2006), in large cross-sectional samples (Soto et al., 2011) and across cultures (Bleidorn et al., 2013). The leading explanations, intrinsic maturation (McCrae et al., 2000) and investment in adult roles (Roberts et al., 2005), are stated for Big Five domains. Read as claims that a domain develops as a single unit, they imply a domain-general view: age acts on a domain as a whole, and its items differ with age only because they measure it. Neither theory, however, is stated at the level of a measurement model, and facet-level versions are as natural (Roberts et al., 2006).

Evidence from below the domain is hard to reconcile with this view. Facets of one domain follow different age trends (Soto et al., 2011), items of one scale fail intercept invariance across age groups (Mõttus et al., 2015), and item-specific variance, or nuance, is reliable and partly heritable (McCrae, 2015; McCrae & Mõttus, 2019; Mõttus et al., 2017, 2019). In a 300-item inventory, age predictions from items correlated r = .65 with age, against r = .28 from domains (Mõttus & Rozgonjuk, 2021), a contrast replicated across samples (Hang et al., 2021; see also Olaru et al., 2024). Yet invariance studies usually find domain-level age comparisons defensible (Brandt et al., 2020; Nye et al., 2016; Zimprich et al., 2012). Prediction contrasts cannot settle the matter: a sum score is an equal-weight factor model (McNeish & Wolf, 2020) that items can out-predict even if age acts only through the domain.

A direct test follows from the structural interpretation of a factor (VanderWeele & Vansteelandt, 2022). If age relates to a domain's items only through the domain factor, then for person i and item j

xij=νj+λjηi+εij,ηi=γ agei+ζi,(1)x_{ij} = \nu_j + \lambda_j \eta_i + \varepsilon_{ij}, \qquad \eta_i = \gamma\,\mathrm{age}_i + \zeta_i, \tag{1}

so that Cov(xj,age)=λjγ Var(age)\mathrm{Cov}(x_j, \mathrm{age}) = \lambda_j \gamma\, \mathrm{Var}(\mathrm{age}): item–age correlations must be proportional to the scaled loadings, rj=ρ ℓjr_j = \rho\,\ell_j with ℓj=λj/SD(xj)\ell_j = \lambda_j / SD(x_j). Equation 1 is a multiple-indicators multiple-causes (MIMIC) model without direct effects (Jöreskog & Goldberger, 1975). Differential item functioning with respect to a continuous covariate is usually studied by adding direct effects item by item against anchor items assumed to be invariant (Muthén, 1989; Woods, 2009), as Gallo et al. (1994) did for age and depressive symptoms, or by letting intercepts and loadings vary with the covariate (Bauer, 2017). We ask the prior question, whether any direct effect is needed, with one anchor-free test per domain; it does not locate biased items and depends on the measurement model that supplies the loadings.

Hypotheses and decision rules were written before any model was fitted. H1 (primary): the restriction would be rejected (Holm-adjusted bootstrap p < .05) in at least four of the five domains (refuted if retained in two or more). H2: out of sample, a domain's five items would predict age better than its optimally weighted factor score (95% CI of the difference excluding zero) in most domains, and the five domain scores would recover less than half of the age variance predicted by all 25 items (refuted if item–factor differences were null in most domains). H3: the median cross-half correlation of item departures would exceed the 95th percentile of an age-permutation null. The study was not preregistered.

Method

Participants

We analysed the bfi data set of the R package psych (Revelle, n.d.-c), retrieved from the Rdatasets mirror on October 24, 2026 (Revelle, n.d.-a): a demonstration subset of 2,800 participants in the Synthetic Aperture Personality Assessment (SAPA) web project, collected in spring 2010 (Revelle, n.d.-b; Revelle et al., 2010). We excluded participants younger than 18 (n = 248) or older than 65 (n = 9, too few to estimate age effects) and, for the primary analyses, those missing any item response (n = 334); the missing-data analyses used all 2,543 age-eligible participants. The 2,209 complete cases (mean age = 29.74 years, SD = 10.39, median = 27; 1,323 aged 18–29, 633 aged 30–44, 253 aged 45–65) were 67.04% women; 48.03% reported some college and 31.82% a college degree or more (education missing for 31). Excluded incomplete cases were older on average (M = 31.20).

Sample size was fixed by the archival data, so we report sensitivity. With N = 2,209, an item–age correlation of r = .060 is detectable with 80% power (α = .05, two-tailed). For the restriction test at α = .01, the most stringent Holm threshold, 80% power requires a noncentrality of 16.75, reached by a single item's direct age effect of r = .067 (neuroticism) to .084 (openness; Table 2).

Materials

The 25 items come from the International Personality Item Pool (IPIP; Goldberg, 1999; Goldberg et al., 2006), five per domain, rated from 1 (very inaccurate) to 6 (very accurate). They are a demonstration selection of IPIP items, not a validated short Big Five inventory, and not John's Big Five Inventory despite the data set's name (Revelle, n.d.-b). Negatively keyed items (A1, C4, C5, E1, E2, O2, O5) were rescored as 7 − x; domain scores are the means of the five keyed items (stems in Table 3). Gender was coded 1 (male) or 2 (female), education in five bands and age in years.

Procedure

Self-selected visitors to the SAPA website answered items sampled from the IPIP (Revelle et al., 2010); Condon et al. (2017) describe the SAPA design and larger public data releases. We used only the public bfi file.

Analysis Plan

For each domain we fitted Equation 1 by maximum likelihood, conditional on age (decades, centred), against an alternative with free item-specific age slopes and the same one-factor residual structure (likelihood-ratio test, 4 df). Items were treated as continuous, which performs well with six categories (Rhemtulla et al., 2012); p values came from a Bollen–Stine-type bootstrap (Bollen & Stine, 1992; 1,999 draws) and were Holm-adjusted across domains (Holm, 1979). The non-proportional share (NPS),

NPS=1−(ℓ⊤r)2(ℓ⊤ℓ)(r⊤r),(2)\mathrm{NPS} = 1 - \frac{(\boldsymbol{\ell}^{\top}\mathbf{r})^{2}}{(\boldsymbol{\ell}^{\top}\boldsymbol{\ell})(\mathbf{r}^{\top}\mathbf{r})}, \tag{2}

is the share of a domain's item–age correlation profile r\mathbf{r} orthogonal to its scaled loadings ℓ\boldsymbol{\ell}, 0 under Equation 1. Item deviations are Δrj=rj−ρ^ ℓj\Delta r_j = r_j - \hat{\rho}\,\ell_j, with ρ^=ℓ⊤r/ℓ⊤ℓ\hat{\rho} = \boldsymbol{\ell}^{\top}\mathbf{r} / \boldsymbol{\ell}^{\top}\boldsymbol{\ell}. Confidence intervals (CIs) come from 999 nonparametric bootstrap resamples, from which we added in revision a simultaneous 95% band for the 25 deviations.

Robustness checks (499 bootstrap draws each) used spline age curves (Harrell, 2015), adjustment for gender, education and acquiescence (Danner et al., 2015; Soto et al., 2008), one correlated uniqueness and a loading-free test of a shared age curve. In revision we added tests on positively keyed items only, under measurement structures respecified until the root-mean-square error of approximation (RMSEA) was at most .08, and two missing-data analyses of the 2,543 age-eligible participants: full-information maximum likelihood (FIML) on all of them and an available-case analysis (details in the Supplemental Method).

For H2, 10-fold cross-validation repeated 10 times predicted age by ordinary least squares from unit-weighted scores, training-fold Bartlett factor scores or items, with 95% CIs from 2,000 participant bootstrap resamples (de Rooij & Weeda, 2020); under Equation 1, items and factor score predict equally well, and factor versus unit score isolates the cost of equal weighting (McNeish & Wolf, 2020). For H3, deviations estimated in 100 random half-splits were correlated across halves, against a null from permuting age within halves. A keying contrast was added after seeing the item pattern (exploratory). No outliers were removed; all responses lay on the 1–6 scale.

Transparency and Openness

We report all exclusions and measures; the data are public (Revelle, n.d.-a). Hypotheses, decision rules, filters and planned analyses were written down before any model was fitted but not preregistered, and the script was developed on these data, so the confirmatory run was not blind to them. Analyses added in revision were requested in peer review and run in an extended script that reproduces every original number exactly. The executed script, its complete numerical output and the data provenance record (source URL, retrieval time, SHA-256 checksum) are published with this article's online run record. We used Python 3.12.13, NumPy 2.5.2, SciPy 1.18.1, pandas 3.0.5 and scikit-learn 1.9.0 (seed 20261024). Values are rounded half up from the four-decimal values in the output; every effect has a 95% CI (Cumming, 2014).

Results

Measurement Models and Domain-Level Age Differences

By Hu and Bentler's (1999) cutoffs no one-factor model fitted closely (Table 1). By the criterion used for respecification (RMSEA ≤ .08), fit was acceptable for agreeableness, extraversion and openness and poor for conscientiousness (RMSEA = .105) and neuroticism (RMSEA = .168), as is common for short Big Five scales (Marsh et al., 2013). Age associations were small: positive for agreeableness and conscientiousness and negative for neuroticism, as usually reported for young and middle adulthood, and positive for extraversion and openness.

Table 1. Measurement Models and Domain-Level Age Associations

Domainαωχ²(5)CFIRMSEASRMRr (score, age) [95% CI]Latent r (age) [95% CI]
Agreeableness.71.7175.44.966.080.033.156 [.115, .196].144 [.094, .192]
Conscientiousness.73.74127.43.942.105.041.087 [.045, .128].106 [.055, .153]
Extraversion.77.7763.62.977.073.028.067 [.026, .109].076 [.030, .127]
Neuroticism.82.81314.68.921.168.058−.098 [−.139, −.056]−.104 [−.150, −.056]
Openness.62.6367.62.947.075.033.057 [.015, .098].074 [.025, .127]

Note. N = 2,209. Fit is for the one-factor model of the five keyed items with age partialled. CFI = comparative fit index; RMSEA = root-mean-square error of approximation; SRMR = standardized root-mean-square residual. Score correlations use Fisher-z CIs; latent correlations come from Equation 1 with nonparametric bootstrap CIs (999 resamples).

H1: Do Domains Carry Their Items' Age Information?

The domain-general restriction was rejected for agreeableness, χ²(4) = 45.15, conscientiousness, χ²(4) = 23.82, and extraversion, χ²(4) = 53.59 (bootstrap ps < .001), and for neuroticism, χ²(4) = 17.32, p = .004, all surviving Holm correction (ps ≤ .007), but not for openness, χ²(4) = 8.37, p = .073 (Table 2). H1 was supported by its prespecified rule. The NPS ranged from .16, 95% CI [.06, .45], for neuroticism to .57 [.28, .92] for extraversion, each beyond the 95th percentile of its null distribution but imprecise; for openness, .28 [.09, .84] fell inside its null distribution.

The rejections did not depend on the poorly fitting models. Freeing the C1–C2 residual covariance gave conscientiousness an acceptable structure (RMSEA = .075), under which the restriction was still rejected, χ²(4) = 22.09, p = .004; neuroticism needed all three residual covariances among N3, N4 and N5 (RMSEA = .000), and its restriction was rejected, χ²(4) = 22.83, p = .002 (fit of both models for every structure in Table S2). Nor did they depend on listwise deletion: with all 2,543 age-eligible participants, all four rejections stood. Openness, however, was then also rejected, χ²(4) = 12.55, p = .014, as it was in its 2,482 available cases, χ²(4) = 11.72, bootstrap p = .024 (Table S3).

Table 2. Tests of the Domain-General Restriction

Domainχ²(4)p (bootstrap)p (Holm)NPS [95% CI]Null NPS 95th pct.Detectable direct effectAcceptable-fit model χ²(4), pAll age-eligible (FIML) χ²(4), p
Agreeableness45.15.0005.0025.23 [.13, .44].09.07745.15, .00254.33, < .001
Conscientiousness23.82.0005.0025.26 [.10, .65].19.07922.09, .00423.69, < .001
Extraversion53.59.0005.0025.57 [.28, .92].34.07653.59, .00267.67, < .001
Neuroticism17.32.0035.0070.16 [.06, .45].12.06722.83, .00217.41, .002
Openness8.37.0725.0725.28 [.09, .84].52.0848.37, .07012.55, .014

Note. N = 2,209, except N = 2,543 in the last column. Primary bootstrap p values (1,999 Bollen–Stine draws) are given to four decimals because their resolution is .0005, the smallest attainable value. NPS = non-proportional share (Equation 2), with the 95th percentile of its distribution under Equation 1. Detectable direct effect = median, across the domain's items, of the single-item direct age effect (correlation metric) detectable with 80% power at α = .01 (noncentrality 16.75; 11.94 at α = .05). Acceptable-fit model: residual covariances freed until RMSEA ≤ .08 (C1–C2 for conscientiousness; N4–N5, N3–N4 and N3–N5 for neuroticism; none needed elsewhere), 499 bootstrap draws (smallest attainable p = .002), Holm-adjusted ps = .010 for the first four domains and .070 for openness. FIML = full-information maximum likelihood, asymptotic p values; Holm-adjusted ps < .001 for agreeableness, conscientiousness and extraversion, .003 for neuroticism and .014 for openness.

All four rejections also survived spline age curves, adjustment for gender, education and acquiescence, a correlated uniqueness and positively keyed items only (ps ≤ .014; Table S1). The loading-free test, however, rejected a shared age curve only for neuroticism, χ²(8) = 22.67, p = .004 (agreeableness, p = .210; conscientiousness, p = .340; extraversion, p = .560): outside neuroticism the departures concern the strength of item–age associations relative to the loadings, and no difference in the shape of the age curves was detected. For openness, the three other loading-based checks gave ps = .036 to .170, the loading-free test p = .208 and positively keyed items p = .456.

Which Items Depart From Their Domains?

Thirteen of the 25 deviations had 95% CIs excluding zero, but only eight survived the simultaneous band, the same eight that survived Holm correction (Table 3): A1, A3, C2, C4, E2, E3, E4 and E5; no neuroticism or openness item did. "Take charge" rose with age more than its loading implied, Δr = .060, band [.007, .113], whereas "Know how to captivate people" and "Make friends easily" showed no detectable age association (95% CIs including zero and excluding the implied value); "Know how to comfort others", agreeableness's best indicator (λ = .768), rose less than implied, Δr = −.070, band [−.108, −.031]. Three of the eight are reverse-keyed items that deviated upward (A1, C4, E2). Neuroticism deviations such as "Panic easily" (−.047) and "Often feel blue" (.046) did not survive adjustment and are exploratory.

Table 3. Item–Age Correlations and Departures From the Domain-Implied Values

ItemStemλr [95% CI]Implied rΔr [95% CI]Simultaneous 95% band
A1 (R)Am indifferent to the feelings of others.361.142 [.101, .182].058.084 [.049, .119][.028, .140]
A2Inquire about others' well-being.656.107 [.065, .148].105.002 [−.026, .029][−.041, .044]
A3Know how to comfort others.768.054 [.012, .095].123−.070 [−.092, −.044][−.108, −.031]
A4Love children.487.122 [.080, .163].078.044 [.011, .078][−.008, .095]
A5Make people feel at ease.625.102 [.060, .143].100.002 [−.028, .030][−.043, .047]
C1Am exacting in my work.545.072 [.031, .114].056.017 [−.017, .051][−.036, .070]
C2Continue until everything is perfect.608−.001 [−.043, .041].062−.063 [−.096, −.031][−.114, −.012]
C3Do things according to a plan.557.052 [.011, .094].057−.005 [−.034, .023][−.051, .042]
C4 (R)Do things in a half-way manner.687.117 [.076, .158].070.047 [.022, .075][.007, .088]
C5 (R)Waste my time.593.060 [.018, .101].061−.001 [−.031, .029][−.048, .046]
E1 (R)Don't talk a lot.594.053 [.011, .095].043.010 [−.018, .039][−.035, .055]
E2 (R)Find it difficult to approach others.737.109 [.067, .150].054.055 [.031, .083][.015, .096]
E3Know how to captivate people.600−.021 [−.062, .021].044−.065 [−.094, −.034][−.111, −.018]
E4Make friends easily.694−.006 [−.047, .036].051−.056 [−.084, −.026][−.101, −.012]
E5Take charge.531.099 [.057, .140].039.060 [.025, .092][.007, .113]
N1Get angry easily.825−.074 [−.115, −.033]−.088.014 [−.010, .035][−.021, .049]
N2Get irritated easily.803−.082 [−.123, −.040]−.086.004 [−.022, .031][−.036, .044]
N3Have frequent mood swings.720−.101 [−.142, −.059]−.077−.024 [−.047, −.0004][−.060, .012]
N4Often feel blue.556−.013 [−.055, .029]−.060.046 [.015, .077][−.002, .094]
N5Panic easily.496−.100 [−.141, −.059]−.053−.047 [−.083, −.014][−.100, .005]
O1Am full of ideas.541.041 [−.001, .082].040.001 [−.033, .032][−.049, .050]
O2 (R)Avoid difficult reading material.495.035 [−.007, .076].037−.002 [−.033, .030][−.052, .048]
O3Carry the conversation to a higher level.633.026 [−.015, .068].047−.020 [−.047, .007][−.064, .024]
O4Spend time reflecting on things.289−.007 [−.049, .035].021−.028 [−.066, .010][−.085, .029]
O5 (R)Will not probe deeply into a subject.547.080 [.038, .121].040.040 [.008, .069][−.007, .087]

Note. N = 2,209. Stems are verbatim from the data set's documentation, with the terminal period omitted; (R) = reverse-keyed, analysed after rescoring. λ = standardized loading in the age-partialled one-factor model. r = item–age correlation (Fisher-z CI). Implied r = ρ^ ℓj\hat{\rho}\,\ell_j, the correlation expected if age acted only through the domain factor. Δr = r − implied r, with nonparametric bootstrap 95% CIs (999 resamples) unadjusted for multiplicity; one bound is shown to four decimals to keep its sign. Simultaneous band = Δr ± 3.05 bootstrap standard deviations, 3.05 being the 95th percentile of the largest absolute studentized deviation across the 25 items (Monte Carlo 95% interval 2.99–3.12); bands excluding zero are in bold, and the same eight items have Holm-adjusted p < .05. Covariate- and acquiescence-adjusted deviations are in Table S4.

H2: Out-of-Sample Prediction

The prespecified per-domain criterion failed: the items beat the factor score reliably only for agreeableness, ΔR² = .016 [.005, .027], and extraversion, ΔR² = .020 [.007, .033], with intervals including zero for the other three domains (Table 4), and the share of the items' out-of-sample R² recovered by the five domain scores, .39 [.18, .58], includes one half. H2 was not supported. The evidence for items came mainly from the aggregate contrast: all 25 items predicted age with R² = .067 [.043, .089], the five domain scores with R² = .026 [.010, .040], ΔR² = .041 [.022, .060]. Factor scores predicted slightly worse than unit scores, ΔR² = −.007 [−.011, −.003], in line with Widaman and Revelle's (2023) defense of sum scores and suggesting that the age information is not aligned with the common factor.

Table 4. Cross-Validated Prediction of Age

PredictorsUnit-weighted score(s) R²Factor score(s) R²Items R²Items − factor ΔR²Factor − unit ΔR²
Agreeableness.022 [.009, .034].014 [.003, .023].030 [.014, .044].016 [.005, .027]−.009 [−.012, −.005]
Conscientiousness.005 [−.003, .012].006 [−.003, .014].014 [.001, .025].007 [−.002, .016].001 [.000, .002]
Extraversion.002 [−.004, .008].002 [−.004, .008].022 [.009, .035].020 [.007, .033].000 [−.001, .001]
Neuroticism.007 [−.002, .016].007 [−.002, .015].013 [.001, .023].006 [−.002, .013].000 [−.002, .002]
Openness.001 [−.005, .005].001 [−.004, .006].001 [−.007, .007]−.001 [−.006, .004].001 [−.001, .002]
All five domains.026 [.010, .040].019 [.005, .031].067 [.043, .089].048 [.028, .067]−.007 [−.011, −.003]

Note. N = 2,209. Out-of-sample R2=1−SSerror/SStotalR^2 = 1 - SS_{\mathrm{error}}/SS_{\mathrm{total}} from 10-fold cross-validation repeated 10 times and averaged over repetitions, with 95% CIs from 2,000 participant bootstrap resamples of out-of-fold errors, conditional on the fold models; negative values mean worse than predicting the mean. For all five domains, items − unit scores ΔR² = .041 [.022, .060]; the share of the items' R² recovered was .39 [.18, .58] by unit scores and .28 [.08, .46] by factor scores; predicted–observed correlations were .16 (unit scores) and .26 (items).

H3: Replicability and Response Style

Item deviations from independent halves correlated at a median r = .78 across 100 splits (2.5th–97.5th percentiles .65–.89), above the 95th percentile of the age-permutation null (.37) in every split (Table S5); H3 was supported. In exploratory analyses, reverse-keyed items deviated more positively than positively keyed items, by .047 [.029, .067], and by .020 [.007, .035] after acquiescence, gender and education were partialled out. The adjustment also moved some deviations the other way (A4 from .044 to .080; O2 from −.002 to −.038), plausibly because the index, built from responses to other domains' items, carries trait and age variance as well as response style (Table S4).

Discussion

In 25 IPIP items, the age differences of four Big Five domains were not domain-general: item–age associations were not proportional to the loadings, the departures replicated across halves, and the rejections held under acceptably fitting measurement models, with positively keyed items, after adjustment for demographics and acquiescence, and with all age-eligible participants. The qualification is the measurement model. Only for neuroticism did a loading-free test also reject, so for agreeableness, conscientiousness and extraversion the evidence is conditional on the estimated loadings and on their invariance across age.

The eight deviations that survived multiplicity adjustment were small, .047 to .084 in absolute value, but so are the domain-level age differences (.06 to .16). The predictive gain was much smaller than in a 300-item inventory (predicted–observed r = .26 for items vs. .16 for domain scores, against .65 vs. .28; Mõttus & Rozgonjuk, 2021) and reliable within only two domains, so H2 failed by its own rule; plausibly, five items per domain carry less item-specific information than long inventories, and a young sample limits age variance.

Maturation and social-investment theories do not themselves imply proportional item differences; the implication follows only if they are read as claims about the domain as a single developing unit (Equation 1), and that reading failed in four domains. Facet-level versions of these accounts are compatible with the rejections. In extraversion, "Take charge" rose more than implied while two positively keyed social items showed no detectable rise, resembling the divergence of social dominance and social vitality (Roberts et al., 2006); in neuroticism, acceptable fit required a shared residual component for mood swings, feeling blue and panic, although no single neuroticism item survived adjustment. Five items per domain cannot separate facet-level from item-level specificity.

The results help reconcile the invariance and nuance literatures. Invariance studies ask whether item-level departures bias domain mean comparisons and often find they do not (Brandt et al., 2020; Nye et al., 2016; Zimprich et al., 2012); here such departures were systematic, so domain-level age differences can be interpretable yet incomplete. The omnibus test is inexpensive and can precede item-wise MIMIC or moderated factor models that locate the departures (Bauer, 2017; Woods, 2009).

Openness was undetermined rather than domain-general: its age association was the weakest (r = .057), its scale the least reliable (α = .62), its test the least sensitive (detectable direct effect .084), and its result depended on specification and on the handling of missing data (ps = .014 to .456). The 25-item complete-case filter served the whole-inventory analyses, whereas each domain's primary test needs only that domain's items; it dropped 273 participants who had answered all five openness items, and with them included the restriction was rejected for openness (available cases, p = .024; FIML, p = .014).

Limitations remain. The data are cross-sectional, so age is confounded with cohort (Rohrer, 2018; Schaie, 1965). The sample was a self-selected 2010 demonstration subset of SAPA, 67.04% women and young (median age 27; 253 participants aged 45–65). Self-reports leave open age differences in item interpretation, social desirability (Ausmees et al., 2022) or response style; the acquiescence index removes substantive as well as stylistic variance, and positively keyed items cannot remove acquiescence that raises them all. The respecified structures came from a data-driven search; items were treated as continuous; FIML assumed responses missing at random; prediction intervals condition on the fold models; some content may be age-graded ("Love children" may track parenthood); and there was no external replication, although larger public SAPA releases would permit one (Condon et al., 2017).

Constraints on Generality. The conclusions apply to adults aged 18 to 65 who volunteered to answer English-language IPIP items online in 2010, to self-reports on these 25 items, a demonstration selection rather than a validated short inventory, and to cross-sectional age differences. We expect the qualitative result, item–age associations not proportional to loadings in most domains, to hold for other item sets, because it parallels findings from long inventories in several countries (Hang et al., 2021); specific items, magnitudes and the openness result may not generalize. We have no evidence on adults over 65, informant reports or longitudinal change.

References

Ausmees, L., Kandler, C., Realo, A., Allik, J., Borkenau, P., Hřebíčková, M., & Mõttus, R. (2022). Age differences in personality traits and social desirability: A multi-rater multi-sample study. Journal of Research in Personality, 99, Article 104245. https://doi.org/10.1016/j.jrp.2022.104245

Bauer, D. J. (2017). A more general model for testing measurement invariance and differential item functioning. Psychological Methods, 22(3), 507–526. https://doi.org/10.1037/met0000077

Bleidorn, W., Klimstra, T. A., Denissen, J. J. A., Rentfrow, P. J., Potter, J., & Gosling, S. D. (2013). Personality maturation around the world: A cross-cultural examination of social-investment theory. Psychological Science, 24(12), 2530–2540. https://doi.org/10.1177/0956797613498396

Bleidorn, W., Schwaba, T., Zheng, A., Hopwood, C. J., Sosa, S. S., Roberts, B. W., & Briley, D. A. (2022). Personality stability and change: A meta-analysis of longitudinal studies. Psychological Bulletin, 148(7–8), 588–619. https://doi.org/10.1037/bul0000365

Bollen, K. A., & Stine, R. A. (1992). Bootstrapping goodness-of-fit measures in structural equation models. Sociological Methods & Research, 21(2), 205–229. https://doi.org/10.1177/0049124192021002004

Brandt, N. D., Becker, M., Tetzner, J., Brunner, M., Kuhl, P., & Maaz, K. (2020). Personality across the lifespan: Exploring measurement invariance of a short Big Five Inventory from ages 11 to 84. European Journal of Psychological Assessment, 36(1), 162–173. https://doi.org/10.1027/1015-5759/a000490

Condon, D. M., Roney, E., & Revelle, W. (2017). A SAPA project update: On the structure of phrased self-report personality items. Journal of Open Psychology Data, 5(1), Article 3. https://doi.org/10.5334/jopd.32

Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7–29. https://doi.org/10.1177/0956797613504966

Danner, D., Aichholzer, J., & Rammstedt, B. (2015). Acquiescence in personality questionnaires: Relevance, domain specificity, and stability. Journal of Research in Personality, 57, 119–130. https://doi.org/10.1016/j.jrp.2015.05.004

de Rooij, M., & Weeda, W. (2020). Cross-validation: A method every psychologist should know. Advances in Methods and Practices in Psychological Science, 3(2), 248–263. https://doi.org/10.1177/2515245919898466

Gallo, J. J., Anthony, J. C., & Muthén, B. O. (1994). Age differences in the symptoms of depression: A latent trait analysis. Journal of Gerontology, 49(6), P251–P264. https://doi.org/10.1093/geronj/49.6.P251

Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality psychology in Europe (Vol. 7, pp. 7–28). Tilburg University Press.

Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006). The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 40(1), 84–96. https://doi.org/10.1016/j.jrp.2005.08.007

Hang, Y., Soto, C., Speyer, L. G., Haring, L., Lee, B., Ostendorf, F., & Mõttus, R. (2021). Age differences in the personality hierarchy: A multi-sample replication study across the life span. Journal of Research in Personality, 93, Article 104121. https://doi.org/10.1016/j.jrp.2021.104121

Harrell, F. E., Jr. (2015). Regression modeling strategies (2nd ed.). Springer. https://doi.org/10.1007/978-3-319-19425-7

Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70. https://www.jstor.org/stable/4615733

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

Jöreskog, K. G., & Goldberger, A. S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70(351a), 631–639. https://doi.org/10.1080/01621459.1975.10482485

Marsh, H. W., Nagengast, B., & Morin, A. J. S. (2013). Measurement invariance of Big-Five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and la dolce vita effects. Developmental Psychology, 49(6), 1194–1218. https://doi.org/10.1037/a0026913

McCrae, R. R. (2015). A more nuanced view of reliability: Specificity in the trait hierarchy. Personality and Social Psychology Review, 19(2), 97–112. https://doi.org/10.1177/1088868314541857

McCrae, R. R., Costa, P. T., Jr., Ostendorf, F., Angleitner, A., Hřebíčková, M., Avia, M. D., Sanz, J., Sánchez-Bernardos, M. L., Kusdil, M. E., Woodfield, R., Saunders, P. R., & Smith, P. B. (2000). Nature over nurture: Temperament, personality, and life span development. Journal of Personality and Social Psychology, 78(1), 173–186. https://doi.org/10.1037/0022-3514.78.1.173

McCrae, R. R., & Mõttus, R. (2019). What personality scales measure: A new psychometrics and its implications for theory and assessment. Current Directions in Psychological Science, 28(4), 415–420. https://doi.org/10.1177/0963721419849559

McNeish, D., & Wolf, M. G. (2020). Thinking twice about sum scores. Behavior Research Methods, 52(6), 2287–2305. https://doi.org/10.3758/s13428-020-01398-0

Mõttus, R., Kandler, C., Bleidorn, W., Riemann, R., & McCrae, R. R. (2017). Personality traits below facets: The consensual validity, longitudinal stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 112(3), 474–490. https://doi.org/10.1037/pspp0000100

Mõttus, R., Realo, A., Allik, J., Esko, T., Metspalu, A., & Johnson, W. (2015). Within-trait heterogeneity in age group differences in personality domains and facets: Implications for the development and coherence of personality traits. PLOS ONE, 10(3), Article e0119667. https://doi.org/10.1371/journal.pone.0119667

Mõttus, R., & Rozgonjuk, D. (2021). Development is in the details: Age differences in the Big Five domains, facets, and nuances. Journal of Personality and Social Psychology, 120(4), 1035–1048. https://doi.org/10.1037/pspp0000276

Mõttus, R., Sinick, J., Terracciano, A., Hřebíčková, M., Kandler, C., Ando, J., Mortensen, E. L., Colodro-Conde, L., & Jang, K. L. (2019). Personality characteristics below facets: A replication and meta-analysis of cross-rater agreement, rank-order stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 117(4), e35–e50. https://doi.org/10.1037/pspp0000202

Muthén, B. O. (1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54(4), 557–585. https://doi.org/10.1007/BF02296397

Nye, C. D., Allemand, M., Gosling, S. D., Potter, J., & Roberts, B. W. (2016). Personality trait differences between young and middle-aged adults: Measurement artifacts or actual trends? Journal of Personality, 84(4), 473–492. https://doi.org/10.1111/jopy.12173

Olaru, G., Jankowsky, K., & Allemand, M. (2024). Structural and inter-individual differentiation in personality traits across the adult lifespan. European Journal of Personality, 38(2), 306–325. https://doi.org/10.1177/08902070231171736

Revelle, W. (n.d.-a). bfi: 25 personality items representing 5 factors [Data set]. psych R package, mirrored by Rdatasets. Retrieved October 24, 2026, from https://vincentarelbundock.github.io/Rdatasets/csv/psych/bfi.csv

Revelle, W. (n.d.-b). Documentation of the bfi data set [Data set documentation]. Rdatasets. https://vincentarelbundock.github.io/Rdatasets/doc/psych/bfi.html

Revelle, W. (n.d.-c). psych: Procedures for psychological, psychometric, and personality research [Computer software]. Northwestern University. https://CRAN.R-project.org/package=psych

Revelle, W., Wilt, J., & Rosenthal, A. (2010). Individual differences in cognition: New methods for examining the personality-cognition link. In A. Gruszka, G. Matthews, & B. Szymura (Eds.), Handbook of individual differences in cognition: Attention, memory, and executive control (pp. 27–49). Springer. https://doi.org/10.1007/978-1-4419-1210-7_2

Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315

Roberts, B. W., Walton, K. E., & Viechtbauer, W. (2006). Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies. Psychological Bulletin, 132(1), 1–25. https://doi.org/10.1037/0033-2909.132.1.1

Roberts, B. W., Wood, D., & Smith, J. L. (2005). Evaluating Five Factor Theory and social investment perspectives on personality trait development. Journal of Research in Personality, 39(1), 166–184. https://doi.org/10.1016/j.jrp.2004.08.002

Rohrer, J. M. (2018). Thinking clearly about correlations and causation: Graphical causal models for observational data. Advances in Methods and Practices in Psychological Science, 1(1), 27–42. https://doi.org/10.1177/2515245917745629

Schaie, K. W. (1965). A general model for the study of developmental problems. Psychological Bulletin, 64(2), 92–107. https://doi.org/10.1037/h0022371

Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2008). The developmental psychometrics of Big Five self-reports: Acquiescence, factor structure, coherence, and differentiation from ages 10 to 20. Journal of Personality and Social Psychology, 94(4), 718–737. https://doi.org/10.1037/0022-3514.94.4.718

Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2011). Age differences in personality traits from 10 to 65: Big Five domains and facets in a large cross-sectional sample. Journal of Personality and Social Psychology, 100(2), 330–348. https://doi.org/10.1037/a0021717

VanderWeele, T. J., & Vansteelandt, S. (2022). A statistical test to reject the structural interpretation of a latent factor model. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(5), 2032–2054. https://doi.org/10.1111/rssb.12555

Widaman, K. F., & Revelle, W. (2023). Thinking thrice about sum scores, and then some more about measurement and analysis. Behavior Research Methods, 55(2), 788–806. https://doi.org/10.3758/s13428-022-01849-w

Woods, C. M. (2009). Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis. Multivariate Behavioral Research, 44(1), 1–27. https://doi.org/10.1080/00273170802620121

Zimprich, D., Allemand, M., & Lachman, M. E. (2012). Factorial structure and age-related psychometrics of the MIDUS personality adjective items across the life span. Psychological Assessment, 24(1), 173–186. https://doi.org/10.1037/a0025265

Supplemental Material

Supplemental Method

Primary test. The Bollen–Stine-type bootstrap rotates the alternative model's residuals to the covariance implied by Equation 1 and regenerates responses from its fitted means; Table S1 gives each null distribution's 95th percentile beside the asymptotic p value. The custom maximum-likelihood estimates agreed with scikit-learn's factor analysis and with an independent fit of the joint covariance structure (Table S1, note).

Simultaneous band and Holm correction. The band for the 25 deviations is each deviation ± c bootstrap standard deviations, c being the 95th percentile, across the 999 resamples, of the largest absolute studentized deviation (Table 3, note). Holm-adjusted p values used the bootstrap z ratio, the deviation divided by its bootstrap standard deviation.

Robustness checks, each with its own 499-draw bootstrap null generated under its null model: (a) age as a restricted cubic spline with four knots at the 5th, 35th, 65th and 95th age percentiles (Harrell, 2015), testing proportionality of whole age curves (12 df); (b) gender, education (dummies with a missing category) and a leave-domain-out acquiescence index partialled out with free direct effects, the index being the mean unreversed response to the balanced positively and negatively keyed items of the other domains (A1, A2, C1, C2, C4, C5, E1–E4, O1, O2, O3 and O5, omitting the tested domain's items; Danner et al., 2015; Soto et al., 2008); (c) a correlated uniqueness for the item pair with the largest standardized residual correlation; (d) a loading-free test that the five items share one age curve, that is, that the second and third canonical correlations between the items and the spline basis are zero (Bartlett χ², 8 df; cf. VanderWeele & Vansteelandt, 2022), which is sensitive only to differences in the shape of age curves; and (e, added in revision) positively keyed items only (df = number of items − 1).

Respecified measurement structures (added in revision). Residual covariances were freed one at a time, largest absolute standardized residual first, until the RMSEA of the alternative model was at most .08, with at most three pairs; Equation 1 and the alternative shared the structure, and the test used 499 Bollen–Stine draws.

Loading drift (added in revision). If each SD-scaled loading drifted linearly with age while the intercepts did not, a first-order approximation makes an item's deviation the product of that drift per SD of age, γ⋅SD(age)\gamma \cdot SD(\mathrm{age}) and the skewness of age (1.00 here), so a deviation of .04 would require a drift of 0.27 (agreeableness) to 0.54 (openness) per SD of age and a deviation of .08 one of 0.55 to 1.08, drifts that are large relative to the loadings themselves (Table 3).

Missing data (added in revision). FIML estimated both models on all 2,543 age-eligible participants conditional on age, which every participant reported, assuming item responses missing at random given age and the observed responses; its p values are asymptotic. Applied to the complete cases, the FIML code reproduced the primary statistics (Table S3, note). The available-case analysis used, for each domain, all age-eligible participants who answered its five items, with 499 Bollen–Stine draws from a separate seeded random stream.

Prediction and replication. Bartlett factor-score weights were estimated in the training folds, and bootstrap CIs resample participants' out-of-fold errors given the fold models. In each of 100 random half-splits the 25 deviations were estimated in each half and correlated across halves; the null permuted age within halves.

Table S1. Robustness of the Restriction Tests

Domainp (χ², asymptotic)Null χ² 95th pct.Spline age χ²(12), pAdjusted χ²(4), pCorrelated pair (residual r)With correlated uniqueness χ²(4), pShared age curve χ²(8), pPositively keyed items χ²(df), p
Agreeableness< .0019.2951.83, .00239.20, .002A1–A2 (.097)48.89, .00210.98, .21025.22 (3), .002
Conscientiousness< .00110.1230.45, .01415.97, .008C1–C2 (.093)22.09, .0049.48, .34013.28 (2), .006
Extraversion< .0019.5660.71, .00242.84, .002E3–E5 (.084)57.50, .0026.95, .56031.32 (2), .002
Neuroticism.0029.6738.18, .00216.35, .006N4–N5 (.125)20.67, .00222.67, .00417.32 (4), .004
Openness.0799.2515.76, .1709.89, .036O2–O5 (.075)9.74, .0469.60, .2081.47 (2), .456

Note. N = 2,209. Null χ² 95th pct. = 95th percentile of the 1,999-draw Bollen–Stine null of the primary test. Other p values are bootstrap values from 499 draws (.002 is the smallest attainable). Spline knots at 18, 23, 31 and 51 years. Adjusted = gender, education and acquiescence partialled out with free direct effects. The shared-age-curve test is the loading-free rank-one test on the spline basis. Positively keyed items: A2–A5, C1–C3, E3–E5, O1, O3 and O4 (neuroticism has no reverse-keyed items and is unchanged); Holm-adjusted ps = .010 (agreeableness, extraversion), .012 (conscientiousness, neuroticism) and .456 (openness). Estimator check: across domains, the custom estimates differed from scikit-learn's FactorAnalysis and from an independent joint-covariance fit by at most 2.8 × 10⁻⁷ in standardized loadings and 3.9 × 10⁻¹⁰ in χ².

Table S2. Fit of the Compared Models Under Each Measurement Structure

DomainStructureH0 χ²(df)H0 CFIH0 RMSEAH0 SRMRHA χ²(df)HA CFIHA RMSEAHA SRMR
AgreeablenessOne factor120.59 (9).949.075.03775.44 (5).968.080.028
+ A1–A265.10 (8).974.057.03016.21 (4).994.037.012
ConscientiousnessOne factor151.25 (9).934.085.039127.43 (5).943.105.034
+ C1–C275.42 (8).969.062.02953.33 (4).977.075.024
ExtraversionOne factor117.22 (9).958.074.03563.62 (5).977.073.023
+ E3–E573.27 (8).975.061.02815.77 (4).995.037.010
NeuroticismOne factor332.00 (9).919.128.051314.68 (5).923.168.049
+ N4–N5255.29 (8).938.118.046234.62 (4).942.162.043
+ N4–N5, N3–N4, N3–N523.99 (6).996.037.0171.16 (2)1.000.000.002
OpennessOne factor75.99 (9).944.058.03067.62 (5).948.075.028
+ O2–O531.41 (8).980.036.02121.67 (4).985.045.016

Note. N = 2,209. H0 = Equation 1 (no direct effects); HA = free item-specific age slopes. Fit is for the joint covariance of the five items and age, with CFI computed against the independence model of all six variables; HA's χ² and RMSEA therefore equal those of the age-partialled measurement model (Table 1 for the one-factor structure), whereas its CFI and SRMR differ. Rows with one freed pair are the correlated-uniqueness check (Table S1); for conscientiousness this is also the acceptable-fit structure (standardized residual covariance .16). Neuroticism's acceptable-fit structure needed three pairs (standardized residual covariances .20, .21 and .15); the measurement model's RMSEA was .168, .162, .115 and .000 as pairs were added. NPS under the acceptable-fit structures: conscientiousness .23 [.09, .64] (null 95th percentile .18), neuroticism .17 [.07, .48] (.15). Age-partialled measurement models after gender, education and acquiescence were partialled out (agreeableness, conscientiousness, extraversion, neuroticism and openness): χ²(5) = 58.05, 104.05, 53.26, 312.94 and 44.31; CFI = .972, .953, .981, .916 and .966; RMSEA = .069, .095, .066, .167 and .060; SRMR = .030, .037, .026, .062 and .027.

Table S3. Missing-Data Sensitivity of the Primary Tests

DomainComplete cases (N = 2,209): χ²(4), p, Holm pFIML (N = 2,543): cases with missing itemsFIML χ²(4), p, Holm pAvailable cases: nAvailable cases χ²(4), bootstrap p, Holm p
Agreeableness45.15, < .001, < .0018154.33, < .001, < .0012,46253.99, .002, .010
Conscientiousness23.82, < .001, < .0018223.69, < .001, < .0012,46124.07, .002, .010
Extraversion53.59, < .001, < .0017967.67, < .001, < .0012,46465.57, .002, .010
Neuroticism17.32, .002, .0039517.41, .002, .0032,44816.41, .004, .010
Openness8.37, .079, .0796112.55, .014, .0142,48211.72, .024, .024

Note. Complete-case p values here are asymptotic, for comparability with FIML (bootstrap values in Table 2); by Holm correction four domains are rejected with complete cases, five with FIML and five with available cases. FIML = full-information maximum likelihood conditional on age, assuming responses missing at random given age and the observed responses; applied to the complete cases, it reproduced the complete-case χ² values to within 3.9 × 10⁻¹⁰. Available cases = age-eligible participants who answered all five of the domain's items; 499 Bollen–Stine draws from a separate seeded stream (.002 is the smallest attainable p; Monte Carlo standard error of openness's p = .007).

Table S4. Item Deviations After Adjustment for Gender, Education and Acquiescence

ItemAdjusted Δr [95% CI]Simultaneous 95% band
A1.030 [−.007, .065][−.028, .087]
A2−.002 [−.030, .025][−.046, .042]
A3−.066 [−.091, −.040][−.107, −.026]
A4.080 [.048, .115][.027, .134]
A5.006 [−.022, .036][−.040, .053]
C1.014 [−.020, .047][−.040, .068]
C2−.054 [−.089, −.022][−.108, −.001]
C3−.012 [−.043, .018][−.059, .035]
C4.030 [.004, .056][−.011, .071]
C5.019 [−.011, .048][−.029, .067]
E1−.008 [−.036, .023][−.056, .041]
E2.048 [.023, .074][.008, .089]
E3−.066 [−.095, −.034][−.114, −.017]
E4−.036 [−.065, −.006][−.083, .011]
E5.061 [.025, .095][.005, .117]
N1.024 [.0004, .046][−.012, .060]
N2.004 [−.021, .030][−.037, .046]
N3−.030 [−.054, −.005][−.068, .009]
N4.034 [.001, .066][−.018, .085]
N5−.045 [−.083, −.010][−.101, .012]
O1.030 [−.006, .061][−.022, .081]
O2−.038 [−.068, −.006][−.087, .011]
O3−.009 [−.038, .018][−.056, .037]
O4−.015 [−.053, .023][−.074, .044]
O5.025 [−.003, .055][−.022, .072]

Note. N = 2,209. Deviations after gender, education and acquiescence were partialled out with free direct effects; nonparametric bootstrap 95% CIs (999 resamples) unadjusted for multiplicity, with one bound shown to four decimals to keep its sign; 13 of these CIs exclude zero. Simultaneous band = Δr ± 3.11 bootstrap standard deviations (Monte Carlo 95% interval 3.00–3.20); the six bands excluding zero are in bold, and the same items have Holm-adjusted p < .05.

Table S5. Replication of Item Deviations and Keying Contrast

QuantityEstimateInterval
Cross-half correlation of the 25 deviations (median of 100 splits).78.65–.89 (2.5th–97.5th percentile of splits)
Age-permutation null: median; 95th percentile−.03; .37—
Splits exceeding the null 95th percentile100%—
Spearman–Brown full-sample reliability of deviations.87—
Reverse-keyed minus positively keyed mean deviation, unadjusted.047[.029, .067]
Same contrast after gender, education and acquiescence adjustment.020[.007, .035]

Note. N = 2,209. The keying contrast is exploratory and uses the 20 items of the four domains with both keying directions (7 reverse-keyed, 13 positively keyed), with 95% bootstrap CIs (999 resamples).

6.Review

Attempt 2Done

Automated review: Accept (novelty 3, rigor 4, clarity 4, reproducibility 5).

Accept
Novelty 3/5Rigor 4/5Clarity 4/5Reproducibility 5/5

Refereed by Claude Fable 5.1, a different model from the one that wrote the paper (Claude Opus 5.5).

Round 2, reissued after the two proof additions. The revision meets every required change from the first report. The three analyses I asked for were run and are correct: fit indices for both compared models under every measurement structure, with the restriction still rejected for conscientiousness under an acceptably fitting model (RMSEA = .075; chi-square(4) = 22.09, p = .004) and for neuroticism under a structure equivalent to a facet factor for N3-N5 (RMSEA = .000; chi-square(4) = 22.83, p = .002); a simultaneous band and Holm correction for the 25 item deviations (8 survive, and item-level claims are confined to them); and FIML plus available-case analyses of all 2,543 age-eligible participants, under which the four rejections stand and openness is also rejected (p = .014 and .024), reported honestly as undetermined. H2 is declared not supported by its pre-stated rule, the decision rules are stated, the theoretical implication is properly conditioned, the wording and rounding inconsistencies are fixed, the transparency section names the archive and software versions, the six missing citations are added and resolve, and the framing and display items match the venue. The two additions I suggested at proof are in and both check against the unchanged record: the Supplemental Method reports the executed loading-drift bound (0.27-0.54 per SD of age for a deviation of .04; 0.55-1.08 for .08; skewness 1.00), and the Discussion states that the complete-case filter served the whole-inventory analyses and dropped 273 participants with complete openness items, with whom openness is rejected (p = .024 available cases; p = .014 FIML). Every number I checked in the main text and all nine tables traces to the re-executed record, whose values are unchanged. What remains is intrinsic and disclosed: a loading-based test that is model-conditional outside neuroticism, data-driven respecification, and a single cross-sectional demonstration sample without external replication. I recommend acceptance; scores unchanged.

Strengths

  • Every required change from the first round has been met, by analysis where analysis was asked for and by text elsewhere, and each new analysis checks against the re-executed record. The extended script reproduces every original number exactly (the primary, item, prediction and replication values in the new RESULT_JSON are identical to the first run; the record's own check 'fit_reproduces_table1' is true), and RESULT_JSON is unchanged by the two proof additions.
  • The measurement-model objection is answered with analysis rather than argument. Conscientiousness reaches RMSEA = .075 with C1-C2 freed and the restriction still fails, chi-square(4) = 22.09, bootstrap p = .004; neuroticism reaches RMSEA = .000 only with all three residual covariances among N3, N4 and N5 freed, which is equivalent to an unmodeled facet factor for those items, the very scenario raised in review, and the restriction still fails, chi-square(4) = 22.83, p = .002; Holm rejects 4 of 5 under the respecified structures. Table S2 gives chi-square, CFI, RMSEA and SRMR for both compared models under every structure, as asked.
  • Item-level inference is now multiplicity-controlled: a sup-t simultaneous band (c = 3.05, Monte Carlo interval 2.99-3.12, with an internal coverage check of .95 across resamples) and Holm-adjusted bootstrap p values agree that 8 of 25 deviations survive (A1, A3, C2, C4, E2, E3, E4, E5); the Statement of Relevance and Discussion cite only these, and the neuroticism items are labelled exploratory.
  • Missing data are handled properly: a full-information ML fit on all 2,543 age-eligible participants, shown to reproduce the complete-case statistics to 4 x 10^-10 when given the complete cases, and an available-case bootstrap from a separate seeded stream. The four rejections stand; the new openness rejections (FIML p = .014, available cases p = .024) are reported in full and openness is called undetermined, with the whole range of p across specifications (.014 to .456) in the abstract. The Discussion now also explains that the 25-item complete-case filter served the whole-inventory analyses and dropped 273 participants (2,482 - 2,209, confirmed in the record) who had answered all five openness items.
  • The executed loading-drift bound is now reported in the Supplemental Method and matches the record (skewness of age 1.00; a deviation of .04 requires an SD-scaled loading drift of 0.27 for agreeableness to 0.54 for openness per SD of age, a deviation of .08 one of 0.55 to 1.08), so the Discussion's caveat about loading invariance across age is now quantified.
  • H2 is reported against its pre-stated rule and declared not supported (items beat the factor score in 2 of 5 domains; recovered share .39 [.18, .58] includes one half), with the decision rules for H1, H2 and H3 stated in the Introduction; the aggregate items-versus-scores contrast is presented for what it is.
  • Reporting matches the record under one stated rounding rule (half up from four decimals): every number I checked in the text, Tables 1-4 and Tables S1-S5 traces to RESULT_JSON, including the respecification path (RMSEA .168, .162, .115, .000), the standardized residual covariances (.20, .21, .15), the FIML case counts (81, 82, 79, 95, 61), the available-case n per domain and the Holm-adjusted values in the notes. All 45 references are among the retrieved sources (Goldberg, 1999, is the dataset's own citation) and all 40 DOIs resolve, including the six added.
  • Venue conventions are met: a 150-word abstract, a 148-word Statement of Relevance, four main tables with five supplementary tables and a Supplemental Method, a Transparency and Openness section that discloses the non-blind confirmatory run, software versions and the run record as the archive, and a Constraints on Generality statement that names the item set as a demonstration selection and the sample as the 2010 SAPA demonstration subset.

Weaknesses

  • The evidence for agreeableness, conscientiousness and extraversion remains conditional on the estimated loadings and on their invariance across age; the only loading-free test rejects for neuroticism alone. This is intrinsic to a loading-based test and is stated in the title, abstract and Discussion, and the loading-drift bound now shows how much drift would be needed, but it bounds the claim.
  • The acceptably fitting structures were found by a data-driven search (for neuroticism, three of ten residual covariances), so the respecified tests are confirmatory only of the proportionality restriction, not of the structure; the Limitations say so.
  • The sample is a single, young, self-selected 2010 demonstration subset with no external replication, and the design is cross-sectional; all are disclosed, and the larger public SAPA releases are named as the replication resource.
  • Density: Table 2 has nine columns and the H1 Results paragraph carries the respecification and missing-data results together; the Supplemental Method is terse on how the FIML likelihood is evaluated (per missingness pattern, conditional on age) and on the studentization behind the band. The main text now runs a few words over the venue's guide, which the editor may accept given what the added passages carry.

Required changes

—

Record

§Event timeline

  1. Run createdQuestion: “Are age differences in personality domain-general or nuance-specific? Testing whether Big Five domain scores capture the age-related information carried by their own items' language=en discipline=psychology venue='Psychological Science” · output language: English · models: agent:claude-opus-5-5
  2. Stage started · Literature reviewAttempt 1 · agent:claude-opus-5-5
  3. awaiting_agent · Literature reviewsearch response for literature_review: literature_review-search-1.response.json
  4. Resume requested
  5. Stage started · Literature reviewAttempt 2 · agent:claude-opus-5-5
  6. Search complete · Literature review74 sources · 17,989 characters of notes
  7. awaiting_agent · Literature reviewcomplete_json response for literature_review: literature_review-complete_json-1.response.json
  8. Resume requested
  9. Stage started · Literature reviewAttempt 3 · agent:claude-opus-5-5
  10. Search complete · Literature review74 sources · 17,989 characters of notes
  11. Stage completed · Literature review
  12. Stage started · IdeationAttempt 1 · agent:claude-opus-5-5
  13. awaiting_agent · Ideationcomplete_json response for ideation: ideation-complete_json-1.response.json
  14. Resume requested
  15. Stage started · IdeationAttempt 2 · agent:claude-opus-5-5
  16. Awaiting approval · Ideation
  17. Approved · IdeationIdea 1 selected
  18. Stage started · ExperimentationAttempt 1 · agent:claude-opus-5-5
  19. Dataset provisioned · Experimentationbfi_personality_items · 2,800 observations · 29 variables · sha256 68ae71a96c21
  20. awaiting_agent · Experimentationcomplete response for experimentation: experimentation-complete-1.response.txt
  21. Resume requested
  22. Stage started · ExperimentationAttempt 2 · agent:claude-opus-5-5
  23. Dataset provisioned · Experimentationbfi_personality_items · 2,800 observations · 29 variables · sha256 68ae71a96c21
  24. awaiting_agent · Experimentationcomplete response for experimentation: experimentation-complete-1.response.txt
  25. Resume requested
  26. Stage started · ExperimentationAttempt 3 · agent:claude-opus-5-5
  27. Dataset provisioned · Experimentationbfi_personality_items · 2,800 observations · 29 variables · sha256 68ae71a96c21
  28. Experiment round · ExperimentationRound 1: exit 0; result parsed (32.526s)
  29. Stage completed · Experimentation
  30. Stage started · AnalysisAttempt 1 · agent:claude-opus-5-5
  31. awaiting_agent · Analysiscomplete_json response for analysis: analysis-complete_json-1.response.json
  32. Resume requested
  33. Stage started · AnalysisAttempt 2 · agent:claude-opus-5-5
  34. Stage completed · Analysis
  35. Stage started · Paper writingAttempt 1 · agent:claude-opus-5-5
  36. awaiting_agent · Paper writingcomplete response for paper_writing: paper_writing-complete-1.response.txt
  37. Resume requested
  38. Stage started · Paper writingAttempt 2 · agent:claude-opus-5-5
  39. Stage completed · Paper writing
  40. Stage started · ReviewAttempt 1 · agent:claude-opus-5-5
  41. awaiting_agent · Reviewcomplete_json response for review: review-complete_json-1.response.json
  42. Resume requested
  43. Stage started · ReviewAttempt 2 · agent:claude-opus-5-5
  44. Awaiting approval · Review
  45. Sent back for revision · ReviewRe-running from Paper writing: “Referee recommendation: minor_revision. Make these required changes, all from results the script already produced (no new analyses, no new numbers): (1) "State explicitly in the Results and Discussion”
  46. Stage started · Paper writingAttempt 3 · agent:claude-opus-5-5
  47. awaiting_agent · Paper writingcomplete response for paper_writing: paper_writing-complete-1.response.txt
  48. Resume requested
  49. Stage started · Paper writingAttempt 4 · agent:claude-opus-5-5
  50. Stage completed · Paper writing
  51. Stage started · ReviewAttempt 1 · agent:claude-opus-5-5
  52. awaiting_agent · Reviewcomplete_json response for review: review-complete_json-1.response.json
  53. Resume requested
  54. Stage started · ReviewAttempt 2 · agent:claude-opus-5-5
  55. Awaiting approval · Review
  56. Approved · Review
  57. Run completereport=/Users/haodingresearch/Desktop/auto-research/runs/studies-opus55/psychology-en/runs/d0124d4c52b0/REPORT.md
  58. Resume requested
  59. Run completereport=/Users/haodingresearch/Desktop/auto-research/runs/studies-opus55/psychology-en/runs/d0124d4c52b0/REPORT.md
  60. Reopened after sign-off · ReviewReopened after sign-off; re-running from Review: “Independent review: the paper is re-refereed by an independent referee (Claude Fable 5.1) that took no part in writing it.”
  61. Stage started · ReviewAttempt 3 · agent:claude-fable-5-1
  62. awaiting_agent · Reviewcomplete_json response for review: review-complete_json-1.response.json
  63. Resume requested
  64. Stage started · ReviewAttempt 4 · agent:claude-fable-5-1
  65. Awaiting approval · Review
  66. Sent back for revision · ReviewRe-running from Experimentation: “Independent referee (Claude Fable 5.1): major_revision. Required changes: (1) [analysis] Report fit indices (chi-square, CFI, RMSEA, SRMR) for the alternative-model measurement structure after the cor”
  67. Stage started · ExperimentationAttempt 4 · agent:claude-opus-5-5
  68. Dataset provisioned · Experimentationbfi_personality_items · 2,800 observations · 29 variables · sha256 68ae71a96c21
  69. awaiting_agent · Experimentationcomplete response for experimentation: experimentation-complete-1.response.txt
  70. Resume requested
  71. Stage started · ExperimentationAttempt 5 · agent:claude-opus-5-5
  72. Dataset provisioned · Experimentationbfi_personality_items · 2,800 observations · 29 variables · sha256 68ae71a96c21
  73. Experiment round · ExperimentationRound 1: exit 0; result parsed (50.336s)
  74. Stage completed · Experimentation
  75. Stage started · AnalysisAttempt 1 · agent:claude-opus-5-5
  76. awaiting_agent · Analysiscomplete_json response for analysis: analysis-complete_json-1.response.json
  77. Resume requested
  78. Stage started · AnalysisAttempt 2 · agent:claude-opus-5-5
  79. Stage completed · Analysis
  80. Stage started · Paper writingAttempt 5 · agent:claude-opus-5-5
  81. awaiting_agent · Paper writingcomplete response for paper_writing: paper_writing-complete-1.response.txt
  82. Resume requested
  83. Stage started · Paper writingAttempt 6 · agent:claude-opus-5-5
  84. Stage completed · Paper writing
  85. Stage started · ReviewAttempt 1 · agent:claude-fable-5-1
  86. awaiting_agent · Reviewcomplete_json response for review: review-complete_json-1.response.json
  87. Resume requested
  88. Stage started · ReviewAttempt 2 · agent:claude-fable-5-1
  89. Awaiting approval · Review
  90. Sent back for revision · ReviewRe-running from Analysis: “Independent referee (Claude Fable 5.1), round 2: accept. The editor asks for the referee's two optional proof-stage additions: (1) one sentence in the Supplemental Method reporting the loading-drift b”
  91. Stage started · AnalysisAttempt 3 · agent:claude-opus-5-5
  92. awaiting_agent · Analysiscomplete_json response for analysis: analysis-complete_json-1.response.json
  93. Resume requested
  94. Stage started · AnalysisAttempt 4 · agent:claude-opus-5-5
  95. Stage completed · Analysis
  96. Stage started · Paper writingAttempt 7 · agent:claude-opus-5-5
  97. awaiting_agent · Paper writingcomplete response for paper_writing: paper_writing-complete-1.response.txt
  98. Resume requested
  99. Stage started · Paper writingAttempt 8 · agent:claude-opus-5-5
  100. Stage completed · Paper writing
  101. Stage started · ReviewAttempt 1 · agent:claude-fable-5-1
  102. awaiting_agent · Reviewcomplete_json response for review: review-complete_json-1.response.json
  103. Resume requested
  104. Stage started · ReviewAttempt 2 · agent:claude-fable-5-1
  105. Awaiting approval · Review
  106. Approved · Review
  107. Run completereport=/Users/haodingresearch/Desktop/auto-research/runs/studies-opus55/psychology-en/runs/d0124d4c52b0/REPORT.md