Content Validation of a Situational Judgment Test of Manager’s Integrity: A Qualitative Approach

Authors

  • Amna Rafique PhD Scholar, Institute of Applied Psychology, University of the Punjab, Lahore, Pakistan. https://orcid.org/0000-0003-2553-3383
  • Dr Saima Ghazal Associate Professor, Institute of Applied Psychology, University of the Punjab, Lahore, Pakistan.

DOI:

https://doi.org/10.52053/jpap.v3i2.113

Keywords:

Leadership Effectiveness, Manager’s Integrity, Situational Judgement Test, Tool Development

Abstract

Leader’s integrity appears to be a significant predictor of their effectiveness (Moorman et al., 2012), however, little empirical evidence has been found to address this relationship. It may be due, partly, to unavailability of sound assessment tools in this area. Current research describes development and content validation of a situational judgement test to assess manager’s integrity that will have potential benefits over existing unidimensional and Likert-type perceptual measures (Becker, 2005; Craig & Gustafson, 1998; Mills & Boardley,  2017). Ten managers and 7 subordinates from multinational organizations were interviewd for generating context specific siatuations and associated responses. Initially, 31 situations and associated 6-7 responses were extracted from interviews. Two independent rounds of subject matter experts (SMEs) reviews resulted in 22 situations and 3-5 associated responses. Finally, 16 situations and 3-5 associated responses were retained after calculating item content validity index (I-CVI) and scale validity index (S-CVI). The resulting test, labeled the manager’s integrity-SJT, displayed high content validity. Thus, present study advances research in the field of leadership by developing a situational judgements test for assessing manager’s integrity towards their subordinates that is pertinent predictor of leadership effectiveness (Moorman, et al., 2012).

References

Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: Evidence for overlapping traits. Psychological Bulletin, 121(2), 219-245. doi:10.1037/0033-2909.121.2.219 DOI: https://doi.org/10.1037/0033-2909.121.2.219

Arthur Jr, W., Glaze, R. M., Jarrett, S. M., White, C. D., Schurig, I., & Taylor, J. E. (2014). Comparative evaluation of three situational judgment test response formats in terms of construct-related validity, subgroup differences, and susceptibility to response distortion. Journal of Applied Psychology, 99(3), 535-545. https://psycnet.apa.org/doi/10.1037/a0035788 DOI: https://doi.org/10.1037/a0035788

Barrett, P. (2001). Pre-employment integrity testing: Current methods, problems, and solutions. British Computer Society: Information Security Specialist Group. https://www.pbarrett.net/presentations/integrity_doc.pdf

Bass, B. M., & Steidlmeier, P. (1999). Ethics, character, and authentic transformational leadership behavior. The Leadership Quarterly, 10(2), 181-217. https://doi.org/10.1016/S1048-9843(99)00016-8 DOI: https://doi.org/10.1016/S1048-9843(99)00016-8

Bass, B. M., & Stogdill, R. M. (1990). Bass & Stogdill's handbook of leadership: Theory, research, and managerial applications: Simon and Schuster. https://books.google.com.pk/books?hl=en&lr=&id=KxLizZ3aYmUC&oi=fnd&pg=PR3&dq=Bass,+B.+M.,+%26+Stogdill,+R.+M.+(1990).+Bass+%26+Stogdill%27s+handbook+of+leadership:+Theory,+research,+and+managerial+applications:+&ots=FP07oQOtu0&sig=z9PYDo12JXQeBpKbDR3a-BT4EEo&redir_esc=y#v=onepage&q=Bass%2C%20B.%20M.%2C%20%26%20Stogdill%2C%20R.%20M.%20(1990).%20Bass%20%26%20Stogdill's%20handbook%20of%20leadership%3A%20Theory%2C%20research%2C%20and%20managerial%20applications%3A&f=false

Becker, T. E. (1998). Integrity in organizations: Beyond honesty and conscientiousness. Academy of Management Review, 23(1), 154-161. https://doi.org/10.5465/amr.1998.192969 DOI: https://doi.org/10.5465/amr.1998.192969

Becker, T. E. (2005). Development and validation of a situational judgment test of employee integrity. International Journal of Selection and Assessment, 13(3), 225-232. https://doi.org/10.1111/j.1468-2389.2005.00319.x DOI: https://doi.org/10.1111/j.1468-2389.2005.00319.x

Bennis, W., & Nanus, B. (1985). The Strategies for Taking Charge. New York: Harper. http://coachjacksonspages.com/44.pdf

Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74(3), 478-494. https://psycnet.apa.org/buy/1989-31702-001 DOI: https://doi.org/10.1037/0021-9010.74.3.478

Bledow, R., & Frese, M. (2009). A situational judgment test of personal initiative and its relationship to performance. Personnel Psychology, 62(2), 229-258. https://doi.org/10.1111/j.1744-6570.2009.01137.x DOI: https://doi.org/10.1111/j.1744-6570.2009.01137.x

Callinan, M., Robertson, I.T. (2000). Work sample testing. International Journal of Selection and Assessment 8, 248–260. http://dx.doi.org/10.1111/1468- 2389.00154 DOI: https://doi.org/10.1111/1468-2389.00154

Christian, M.S., Edwards, B.D., Bradley, J.C. (2010). Situational judgement tests: constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology 63, 83–117. http://dx.doi.org/10.1111/j.1744- 6570.2009.01163.x DOI: https://doi.org/10.1111/j.1744-6570.2009.01163.x

Cohen, A. R. (1984). Managing for Excellence: the Guide to Developing High Performance in Contemporary Organizations: New York; Toronto: Wiley. http://ecsocman.hse.ru/text/19183292/

Craig, S. B., & Gustafson, S. B. (1998). Perceived leader integrity scale: An instrument for assessing employee perceptions of leader integrity. The Leadership Quarterly, 9(2), 127-145. https://doi.org/10.1016/S1048-9843(98)90001-7 DOI: https://doi.org/10.1016/S1048-9843(98)90001-7

Denison, D. R., Hooijberg, R., & Quinn, R. E. (1995). Paradox and performance: Toward a theory of behavioral complexity in managerial leadership. Organization Science, 6(5), 524-540. https://doi.org/10.1287/orsc.6.5.524 DOI: https://doi.org/10.1287/orsc.6.5.524

Fertig, S. (2009). The Incremental Validity of a Situational Judgement Test (SJT) Relative to Personality and Cognitive Ability to Predict Managerial Performance. Stellenbosch: University of Stellenbosch. http://hdl.handle.net/10019.1/1769

Fisher, D. M., Good, S., Toich, M. J., & Schutt, E. (2021). Measuring moral disengagement with a situational judgment test: Advancing the assessment of an important workplace construct. International Journal of Selection and Assessment, 29(1), 114-133. https://doi.org/10.1111/ijsa.12318 DOI: https://doi.org/10.1111/ijsa.12318

Furr, R. M. (2021). Psychometrics: An Introduction. SAGE publications. http://shc.sbmu.ac.ir/uploads/Dimensionality_and_Factor_Analysis_.pdf

George, B. (2010). True North: Discover your Authentic Leadership (Vol. 143.). John Wiley & Sons. https://books.google.com.pk/books?hl=en&lr=&id=JcE_A1lo7dkC&oi=fnd&pg=PR9&dq=George,+B.+(2010).+True+north:+Discover+your+authentic+leadership+(Vol.+143)&ots=PSc6kNahuu&sig=AJtG_SJwLUgqL8Qdc8x0aOWMQig&redir_esc=y#v=onepage&q&f=false

Haynes, S. N., Richard, D., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238-247. https://psycnet.apa.org/doi/10.1037/1040-3590.7.3.238 DOI: https://doi.org/10.1037/1040-3590.7.3.238

Howell, J. M., & Avolio, B. J. (1995). The ethics of charismatic leadership: submission or liberation? Academy of Management Perspectives, 6(2). https://doi.org/10.5465/ame.1992.4274395 DOI: https://doi.org/10.5465/ame.1992.4274395

Kanter, R. (1985). Supporting innovation and venture development in established companies. Journal of Business Venturing, 1(1), 47-60. https://doi.org/10.1016/0883-9026(85)90006-0 DOI: https://doi.org/10.1016/0883-9026(85)90006-0

Kirkpatick, S. A., & Locke, E. A. (1991). Leadership: do traits matter? Academy of Management Perspectives, 5(2), 48-60. https://doi.org/10.5465/ame.1991.4274679 DOI: https://doi.org/10.5465/ame.1991.4274679

Latham, G. P., & Saari, L. M. (1979). Importance of supportive relationships in goal setting. Journal of Applied Psychology, 64(2), 151-156. https://psycnet.apa.org/doi/10.1037/0021-9010.64.2.151 DOI: https://doi.org/10.1037/0021-9010.64.2.151

Lennick, D., & Kiel, F. (2007). Moral Intelligence: Enhancing Business Performance and Leadership Success. Pearson Prentice Hall. https://books.google.com.pk/books?hl=en&lr=&id=3_JpOX9nF0MC&oi=fnd&pg=PR21&dq=Lennick,+D.,+%26+Kiel,+F.+(2007).+Moral+intelligence:+Enhancing+business+performance+and+leadership+success:&ots=5PTN7cvK23&sig=aN5fXdPegORw-caquaQ9FXAFSSA&redir_esc=y#v=onepage&q=Lennick%2C%20D.%2C%20%26%20Kiel%2C%20F.%20(2007).%20Moral%20intelligence%3A%20Enhancing%20business%20performance%20and%20leadership%20success%3A&f=false

Lievens, F., De Corte, W., & Schollaert, E. (2008). A closer look at the frame-of-reference effect in personality scale scores and validity. Journal of Applied Psychology, 93(2), 268-279. https://psycnet.apa.org/doi/10.1037/0021-9010.93.2.268 DOI: https://doi.org/10.1037/0021-9010.93.2.268

Lievens, F., & Motowidlo, S. J. (2016). Situational judgment tests: From measures of situational judgment to measures of general domain knowledge. Industrial and Organizational Psychology, 9(1), 3-22. https://doi.org/10.1017/iop.2015.71 DOI: https://doi.org/10.1017/iop.2015.71

McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & GRUBB III, W. L. (2007). Situational judgment tests, response instructions, and validity: A meta?analysis. Personnel Psychology, 60(1), 63-91. https://doi.org/10.1111/j.1744-6570.2007.00065.x DOI: https://doi.org/10.1111/j.1744-6570.2007.00065.x

Mills, J. P., & Boardley, I. D. (2017). Development and initial validation of an indirect measure of transformational leadership integrity. Psychology of Sport and Exercise, 32, 34-46. https://doi.org/10.1016/j.psychsport.2017.05.005 DOI: https://doi.org/10.1016/j.psychsport.2017.05.005

Moorman, R. H., Darnold, T. C., Priesemuth, M., & Dunn, C. P. (2012). Toward the measurement of perceived leader integrity: Introducing a multidimensional approach. Journal of Change Management, 12(4), 383-398. https://doi.org/10.1080/14697017.2012.728746 DOI: https://doi.org/10.1080/14697017.2012.728746

Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640- 647. DOI: https://doi.org/10.1037/0021-9010.75.6.640

Nosek, B. A., & Smith, F. L. (2007). A multitrait multimethod validation of the Implicit Association Test: implicit and explicit attitudes are related but distinct constructs. Experimental psychology, 54(1),14. DOI: https://doi.org/10.1027/1618-3169.54.1.14

Palanski, M. E., & Yammarino, F. J. (2007). Integrity and leadership:: clearing the conceptual confusion. European Management Journal, 25(3), 171-184. https://doi.org/10.1016/j.emj.2007.04.006 DOI: https://doi.org/10.1016/j.emj.2007.04.006

Parry, K. W., & Proctor-Thomson, S. B. (2002). Perceived integrity of transformational leaders in organisational settings. Journal of Business Ethics, 35(2), 75-96. https://doi.org/10.1023/A:1013077109223 DOI: https://doi.org/10.1023/A:1013077109223

Peterson, D. (2004). Perceived leader integrity and ethical intentions of subordinates. Leadership & Organization Development Journal, 25(1), 7-23. https://doi.org/10.1108/01437730410512741 DOI: https://doi.org/10.1108/01437730410512741

Peus, C., Braun, S., & Frey, D. (2013). Situation-based measurement of the full range of leadership model, Development and validation of a situational judgment test. The Leadership Quarterly, 24(5), 777-795. https://doi.org/10.1016/j.leaqua.2013.07.006 DOI: https://doi.org/10.1016/j.leaqua.2013.07.006

Podsakoff, P. M., & Todor, W. D. (1985). Relationships between leader reward and punishment behavior and group processes and productivity. Journal of Management, 11(1), 55-73. https://doi.org/10.1177%2F014920638501100106 DOI: https://doi.org/10.1177/014920638501100106

Price, T. L. (2003). The ethics of authentic transformational leadership. The Leadership Quarterly, 14(1), 67-81. https://doi.org/10.1016/S1048-9843(02)00187-X DOI: https://doi.org/10.1016/S1048-9843(02)00187-X

Rockstuhl, T., Ang, S., Ng, K.-Y., Lievens, F., & Van Dyne, L. (2015). Putting judging situations into situational judgment tests: Evidence from intercultural multimedia SJTs. Journal of Applied Psychology, 100(2), 464-480. https://psycnet.apa.org/doi/10.1037/a0038098 DOI: https://doi.org/10.1037/a0038098

Seltzer, J., & Bass, B. M. (1990). Transformational leadership: Beyond initiation and consideration. Journal of Management, 16(4), 693-703. https://doi.org/10.1177%2F014920639001600403 DOI: https://doi.org/10.1177/014920639001600403

Simons, T. L. (1999). Behavioral integrity as a critical ingredient for transformational leadership. Journal of Organizational Change Management, 12(2), 94-104. https://doi.org/10.1108/09534819910263640 DOI: https://doi.org/10.1108/09534819910263640

Vroom, V. H., & Yetton, P. W. (1973). Leadership and Decision Making. University of Pittsburgh Press, USA. DOI: https://doi.org/10.2307/j.ctt6wrc8r

Yammarino, F. J., & Bass, B. M. (1990). Transformational leadership and multiple levels of analysis. Human Relations, 43(10), 975-995. https://doi.org/10.1177%2F001872679004301003 DOI: https://doi.org/10.1177/001872679004301003

Yukl, G. (2008). How leaders influence organizational effectiveness. The Leadership Quarterly, 19(6), 708-722. https://doi.org/10.1016/j.leaqua.2008.09.008 DOI: https://doi.org/10.1016/j.leaqua.2008.09.008

Yukl, G., & Van Fleet, D. D. (1992). Theory and research on leadership in organizations. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 147–197). Consulting Psychologists Press. https://psycnet.apa.org/record/1993-97201-003

Yusoff, M. S. B. (2019). ABC of content validation and content validity index calculation. Education in Medicine Journal, 11(2), 49–54. https://doi.org/10.21315/eimj2019.11.2.6 DOI: https://doi.org/10.21315/eimj2019.11.2.6

Downloads

Published

2022-06-30

How to Cite

Rafique, A., & Ghazal, D. S. (2022). Content Validation of a Situational Judgment Test of Manager’s Integrity: A Qualitative Approach. Journal of Professional & Applied Psychology, 3(2), 306–318. https://doi.org/10.52053/jpap.v3i2.113

Issue

Vol. 3 No. 2 (2022): Journal of Professional & Applied Psychology

Section

Qualitative & Mixed Method Articles

Categories

License

Copyright (c) 2022 Amna Rafique, Dr Saima Ghazal

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Creative Commons Attribution-Non Commercial 4.0 International (CC BY-NC 4.0) License: This article is distributed under the terms of the Creative Commons Attribution-Non Commercial 4.0 International License (https://www.creativecommons.org/licenses/by-nc/4.0/) which permits non-Commercial use, reproduction
and distribution of the work without further permission provided the original work is attributed as specified on the Journal website at (https://iprpk.com/ojs/index.php/jpap)