Keywords
multiple comparisons, quantitative literacy, online learning, subgroup analyses
Abstract
The more statistical analyses performed in the analysis of research data, the more likely it is that one or more of the conclusions will be in error. Multiple statistical analyses can occur when the sample contains several subgroups and the researchers perform separate analyses for each subgroup. For example, separate analyses may be done for different ethnic groups, different levels of education, and/or for both genders. Media reports of research frequently omit information on the number of subgroup analyses performed thus leaving the reader with insufficient information to assess the validity of the conclusions. This article discusses the problems with a media report on research that was analyzed by conducting many subgroup analyses. The article concludes that the quantitatively literate reader should be skeptical of articles that report subgroup analyses without reporting the number of analyses that were done.
DOI
http://dx.doi.org/10.5038/1936-4660.6.1.7
Recommended Citation
Lane, David M.. "The Problem of Too Many Statistical Tests: Subgroup Analyses in a Study Comparing the Effectiveness of Online and Live Lectures." Numeracy 6, Iss. 1 (2013): Article 7. DOI: http://dx.doi.org/10.5038/1936-4660.6.1.7
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License
Included in
Behavior and Ethology Commons, Bioinformatics Commons, Educational Assessment, Evaluation, and Research Commons, Educational Psychology Commons, Laboratory and Basic Science Research Commons, Management Sciences and Quantitative Methods Commons, Medicine and Health Sciences Commons, Other Food Science Commons, Social and Behavioral Sciences Commons
