A limitation of the Cognitive Reflection Test: familiarity

Stefan Stieger, Ulf-Dietrich Reips

Research output: Journal article (peer-reviewed)Journal article

52 Citations (Scopus)


The Cognitive Reflection Test (CRT; Frederick, 2005) is a frequently used measure of cognitive vs. intuitive reflection. It is also a frequently found entertaining 'test' on the Internet. In a large age-stratified community-based sample (N = 2,272), we analyzed the impact of having already performed the CRT or any similar task in the past. Indeed, we found that 44% of participants had experiences with these tasks, which was reflected in higher CRT scores (Cohen's d = 0.41). Furthermore, experienced participants were different from naïve participants in regard to their socio-demographics (younger, higher educated, fewer siblings, more likely single or in a relationship than married, having no children). The best predictors of a high CRT score were the highest educational qualification (4.62% explained variance) followed by the experience with the task (3.06%). Therefore, we suggest using more recent multi-item CRTs with newer items and a more elaborated test construction.

Original languageEnglish
Article numbere2395
Pages (from-to)e2395
Issue number9
Publication statusPublished - 2016
Externally publishedYes


  • Age-stratified sample
  • Cognitive Reflection Test
  • Cognitive vs. intuitive reflection
  • Dominance analysis
  • Test experience
  • Validity

ASJC Scopus subject areas

  • General Neuroscience
  • General Biochemistry,Genetics and Molecular Biology
  • General Agricultural and Biological Sciences


Dive into the research topics of 'A limitation of the Cognitive Reflection Test: familiarity'. Together they form a unique fingerprint.

Cite this