Statistical conclusion validity
Appearance
Statistical conclusion validity refers to the appropriate use of statistics to infer whether the presumed independent and dependent variables covary (Cook & Campbell, 1979). It concerns two related statistical inferences: (1) whether the presumed cause and effect covary and (2) how strongly they covary.
The most common threats to statistical conclusion validity are:
- Violated assumptions of the test statistics
- Fishing and the error rate problem
- Unreliability of measures
- Restriction of range
- Unreliability of treatment implementation
- Extraneous variance in the experimental setting
- Heterogeneity of the units under study
- Inaccurate effect size estimation
References
- Cohen, R. J., & Swerdlik, M. E. (2004). Psychological testing and assessment (6th edition). Sydney: McGraw-Hill, pg. 161.
- Cook, T. D., Campbell, D. T., & Day, A. (1979). Quasi-experimentation: Design & analysis issues for field settings. Houghton Mifflin Boston.
- Shadish, W., Cook, T. D.,& Campbell, D. T. (2006). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.