Does Excluding Some Groups From Research Designs Improve Statistical Power?

Torsten Biemann*, Heike Heidemeier

*Corresponding author for this work

Research output: Journal article (peer-reviewed)Journal article

14 Citations (Scopus)

Abstract

This article aims to contribute to a controversy over whether excluding some small or incomplete groups from a sample improves statistical power in group research designs (designs that relate group-level characteristics to group-level outcome measures). In a series of simulation studies, we examined the trade-off between lower reliability and smaller sample size that occurs when very small groups, or incomplete groups are excluded. Distinguishing reflective aggregation models (where scores for different group members are interchangeable) and formative aggregation models (where scores for different group members are not interchangeable), we analyzed the impact that the number of groups, the number of individuals within groups, intraclass correlation (ICC[1]) values, and interrater agreement have on statistical power. The results provided evidence that excluding groups is mostly ill-advised and may fail to improve the conclusions that researchers draw from their results. Common practice and the assumptions that researchers make when excluding groups from their samples are discussed.

Original languageEnglish
Pages (from-to)387-409
Number of pages23
JournalSmall Group Research
Volume43
Issue number4
DOIs
Publication statusPublished - Aug 2012
Externally publishedYes

Keywords

  • group size
  • interrater agreement
  • statistical power

ASJC Scopus subject areas

  • Social Psychology
  • Applied Psychology

Fingerprint

Dive into the research topics of 'Does Excluding Some Groups From Research Designs Improve Statistical Power?'. Together they form a unique fingerprint.

Cite this