Three standard tests of a core therapy skill do not measure the same thing

Three standard ways of measuring cognitive reappraisal — the emotional-regulation skill at the core of cognitive behavioural therapy — do not appear to measure the same thing, according to a study published in Nature. An international collaboration compared trait questionnaires, task-based affective ratings, and amygdala down-regulation recorded in 40 functional MRI datasets, and found that almost none of the variance in one showed up in the others.

Reappraisal is not a fringe construct. Cognitive behavioural therapy is described as one of the most successful forms of therapy, and its central act — rethinking a situation until it hurts less — has been linked to emotional resilience in study after study. If the instruments do not converge, the field cannot say who is good at the skill, whether an intervention improved it, or what the brain-imaging literature on reappraisal has been observing all along. The effect sizes were so small that a single study would need 1,105 participants to detect them with 80 percent power; almost no fMRI lab tests anywhere near that number. The brain data behaved oddly as well: regions responsible for high-level decision-making were not engaged during reappraisal as the theory requires. Instead, amygdala activity simply tracked activity elsewhere — stronger emotional responses came with busier reasoning centres, tamped-down responses with quieter ones.

Negative results are nearly impossible to publish; findings of no effect end up in low-profile journals or survive as unpublished lab lore. No career advances by checking whether two questionnaires agree, and every career advances by reporting a new effect measured with one of them. The constructs keep working regardless — in clinics, in grant applications, and in the wider economy of tests and training apps built on psychological measurement.

The result recasts psychology's replication crisis, usually told as a story of misconduct and questionable statistics, as something closer to a metrology failure: fields running on constructs that never demonstrated convergent validity, the property of different instruments agreeing when pointed at the same thing. The month has already produced one retreat of the kind — Uta Frith, one of the researchers who helped establish the autism spectrum, has publicly abandoned the concept as diagnosis numbers multiplied.

The study's authors call for a fundamental reassessment of how reappraisal capacity is measured and interpreted. The questionnaires remain in circulation.