Range restriction, a hidden threat to the criterion validity
Keywords:
range restriction, variability, predictive power, psychological testsAbstract
Range restriction is manifested by reducing artificial variability in the measured values of sample variability relative to the reference population. Its direct effect is to reduce the Pearson correlation coefficient and underestimation of the predictive power of psychological tests. The article analyzes the sources and effects of range restriction, and describes how to correct it.
Downloads
References
Aguinis, H. (1995). Statistical power problems with moderated multiple regression in management research. Journal of Management Research, 21, 1141-1158.
Aguinis, H., Henle, C. A., & Ostroff, C. (2001). Measurement in Work and Organizational Psychology. In N. Anderson, D. S. Ones, H. K. Sinangil & C. Viswesvaran (Eds.), Handbook of Industrial, Work and Organizational Psychology (Vol. 1 - Personnel Psychology, pp. 27-50).
Alexander, R. A., Carson, K. P., Alliger, G. M., & Barrett, G. V. (1984). Correction for Restriction of Range when Both X and Y are Truncated. Applied Psychological Measurement, 8(2), 231-241.
Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and Performance at the Beginning of the New Millennium: What Do We Know and Where Do We Go Next? Personality and Performance, 9(1/2), 9-30.
Bobko, P. (2001). Correlation and Regression. Applications for Industrial Organizational Psychology and Management (Second ed.). Thousand Oaks: Sage Publications.
Bobko, P., Roth, P. L., & Bobko, C. (2001). Correcting the Effect Size of d for Range Restriction and Unreliability. Organizational Research Methods, 4(1), 46-61.
Borack, J. I. (1994). Estimating Predictive Validity When Restriction Due To Range Restriction and Attrition Is Present. Military Psychology, 6(3), 193-204.
Campbell, J. S., Castaneda, M., & Pulos, S. (2010). Meta-Analysis of Personality Assessments as Predictors of Military Aviation Training Success. International Journal of Aviation Psychology, 20(1), 92-109. doi: 10.1080/10508410903415872
Carretta, T. R., & Ree, M. J. (2001). Pitfalls of Ability Research. International Journal of Selection and Assessment, 9(4), 325-335.
Carretta, T. R., & Ree, M. J. (2003). Pilot Selection Methods In P. S. Tsang & M. A. Vidulich (Eds.), Principles and Practice of Aviation Psychology (pp. 357-396). London: Lawrence Erlbaum Associates, Publishers.
Dunbar, S. B., & Linn, R. L. (1991). Range Restriction Adjustments in the Prediction of Military Job Performance. In A. K. Wigdor & B. F. Green Jr. (Eds.), Performance Assessment for the Workplace - Technical issues (Vol. II, pp. 127-157). Washington D.C.
Ercan, I., Yazici, B., Sigirli, D., Ediz, B., & Kan, I. (2007). Examining Cronbach Alpha, Theta, Omega Reliability Coefficients According to the Sample Size. Journal of Modern Applied Statistical Methods, 6(1), 291-303.
Frazier, P. A., Tix, A. P., & Barron, K. E. (2004). Testing Moderator and Mediator Effects in Counseling Psychology Research. Journal of Counseling Psychology, 51(1), 115-134.
Hogan, J. (1998). Personality and Job Performance. Human Performance, 11(2/3), 125.
Hurtz, G. M., & Donovan, J. J. (2000). Personality and Job Performance: The Big Five Revisited. Journal of Applied Psychology, 85(6), 869-879.
Jex, S. M. (2002). Organizational psychology: a scientist-practitioner approach. New York: John Wiley & Sons.
Judge, T. A., Klinger, R., Simon, L. S., & Yang, I. W. F. (2008). The Contributions of Personality to Organizational Behavior and Psychology: Findings, Criticisms, and Future Research Directions. Social and Personality Psychology Compass 2(5), 1982-2000.
Kline, R. B. (2011). Principles and Practice of Structural Equation Modeling (Third ed.). New York: The Guilford Press.
Landy, F. J., & Conte, J. M. (2009). Work in the 21st Century: An Introduction to Industrial and Organizational Psychology (Third ed.): Wiley-Blackwell.
Lang, J. W. B., Kersting, M., & Hülsheger, U. R. (2010). Range Shrinkage of Cognitive Ability Test Scores in Applicant Pools for German Governmental Jobs: Implications for range restriction corrections. International Journal of Selection and Assessment, 18(3), 321-327.
Leaetta, M. H., & Frederick, L. O. (2000). Personnel Selection: Looking Toward the Future - Remembering the Past. Annual Review of Psychology, 51, 631-666.
Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K., & Schmitt, N. (2007). Reconsidering the Use of Personality Tests in Personnel Selection Contexts. Personnel Psychology, 60(3), 683-729. doi: 10.1111/j.1744-6570.2007.00089.x
Muchinsky, P. (2006). Psychology Applied To Work (Eight ed.): Thomson Learning Inc.
Mumford, M. D., & Mendoza, J. e. L. (1983, March, 23-26). Range Restriction and Attenuation Corrections. Paper presented at the Annual Meeting of the Southeastern Psychological Association, Atlanta, GA.
Muthén, B. O., & Hsu, J.-W. Y. (1993). Selection and predictive validity with latent variable structures. British Journal of Mathematical and Statistical Psychology, 46, 255-271.
Nowack, K. (1997). Personality Inventories: The Next Generation. Performance in Practice. American Society of Training and Development (Winter 1996/1997).
Oswald, F. L., & Converse, P. D. (2005). Correcting for Reliability and Range-Restriction in Meta-Analysis. Paper presented at the 20th Annual Conference for the Society for Industrial and Organizational Psychology, Los Angeles, CA.
Pitariu, H. (2003). Proiectarea fișelor de post, evaluarea muncii și aprecierea personalului. București: Irecson.
Ployhart, R. E., & Holtz, B. C. (2008). The diversity-validity dilemma: strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology, 61, 153-172.
Popa, M. (2002). Armstrong Laboratory Aviation Personality Survey Inventory (ALAPS); Romanian Release. Paper presented at the 25-th EAAP Conference, Warsaw.
Pyburn, K. M., Ployhart, R. E., & Kravitz, D. A. (2008). The diversity-validity dilemma: overview and legal context. Personnel Psychology, 61(1), 143-151. doi: 10.1111/j.1744-6570.2008.00108.x
Raju, N. S., & Brand, P. A. (2003). Determining the significance of correlations corrected for unreliability and range restriction. Applied Psychological Measurement, 27(1), 52-71.
Retzlaff, P. D. (2002). The Armstrong Laboratory Aviation Personality Survey: Development, Norming, and Validation. Military Medicine, 167(12).
Retzlaff, P. D., King, R. E., McGlohn, S. E., & Callister, J. D. (1996). The Development of the Armstrong Laboratory Aviation Personality Survey (ALAPS), : Interim Technical Report, Aerospace Medicine Directorate, Clinical Sciences Division, Brooks Air Force Base, TX.
Royston, P., Altman, D. G., & Sauerbrei, W. (2006). Dichotomizing continuous predictors in multiple regression: a bad idea. Statistics in Medicine, 25, 127-141.
Sackett, P. R., & Lievens, F. (2008). Personnel Selection. Annual Review of Psychology, 59:16.1-16.32 (http://users.ugent.be/~flievens/ar.pdf).
Sackett, P. R., Lievens, F., Berry, C. M., & Landers, R. N. (2007). A Cautionary Note on the Effects of Range Restriction on Predictor Intercorrelations. Journal of Applied Psychology 92(2), 538-544.
Sackett, P. R., & Ostgaard, D. J. (1994). Job-specific applicant pools and national norms for cognitive ability tests: Implications for range restriction corrections in validation research. Journal of Applied Psychology, 79(5), 680-684.
Salgado, J. F., Viswesvaran, C., & Ones, D. S. (2005). Predictors Used for Personnel Selection: An Overview of Constructs, Methods and Techniques. In N. Anderson, D. S. Ones, H. K. Sinangil & C. Viswesvaran (Eds.), Handbook of Industrial, Work & Organizational Psychology (Vol. 1 - Personnel Psychology, pp. 165-199). London: Sage Publishing.
Schmidt, F. L., & Hunter, J. (2005). Meta-Analysis. In N. Anderson, D. S. Ones, H. K. Sinangil & C. Viswesvaran (Eds.), Handbook of Industrial, Work & Organizational Psychology (Vol. 1 - Personnel Psychology, pp. 51-70). London: Sage Publishing.
Schmidt, F. L., & Shaffer, J. A. (2008). Increased Accuracy for Range Restriction Corrections: Implications for the Role of Personality and General Mental Ability In Job and Training Performance. Personnel Psychology, 61(4), 827-868.
SIOP. (2003). Principles for the Validation and Use of Personnel Selection Procedures (Fourth ed.): Society for Industrial Organizational Psychology (http://www.siop.org/_Principles/principles.pdf).
Stauffer, J. M., & Mendoza, J. L. (2001). The Proper Sequence for Correcting Correlation Coefficients for Range Restriction and Unreliability. Psychometrika, 66(1), 63-68.
Urbina, S. (2004). Essentials of Psychological Testing. Hoboken: John Wiley & Sons, Inc.
Wells, D. G., & Fruchter, B. (1970). Correcting the Correlation Coefficient for Explicit Restriction on Both Variables. Educational and Psychological Measurement, 30(4), 925-934. doi: 10.1177/001316447003000412
Whetzel, D. L., McDaniel, M. A., Yost, A. P., & Kim, N. (2010). Linearity of Personality–Performance Relationships: A large-scale examination. International Journal of Selection and Assessment, 18(3), 310-320. doi: 10.1111/j.1468-2389.2010.00514.x
Wiberg, M., & Sundström, A. (2009). A comparison of two approaches to correction of restriction of range in correlation analysis. Practical Assessment, Research & Evaluation http://pareonline.net/pdf/v14n5.pdf), 14(5).
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal,it also allows for use of the work for non-commercial purposes and if others remix, transform or build upon the works found in this journal they must distribute the contributions under the same licence as the original.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See: The Effect of Open Access).
