How useful are personality tests when it comes to selecting and developing your people? Dr Rob Yeung looks into the reliability and validity of questionnaires
Studying this technical article and answering the related questions can count towards your verifiable CPD if you are following the unit route to CPD and the content is relevant to your learning and development needs. One hour of learning equates to one unit of CPD. We'd suggest that you use this as a guide when allocating yourself CPD units.
This article was first published in the July/August 2020 China edition of Accounting and Business magazine.
Many organisations use personality tests in both selecting and developing people. However, some questionnaires are much more strongly grounded in scientific evidence than others.
When measuring the usefulness of any test, psychologists look at two properties: reliability and validity. Reliability is the extent to which a test’s results are reproducible; validity is the degree to which it correlates with valuable outcomes in the real world.
Reliability is important because personality is reasonably fixed by early adulthood and only changes gradually with life experience. People may feel more sociable, curious or aggressive on some days than others; however, a personality questionnaire should give us fairly reproducible or reliable results if we were to administer it several weeks or months apart.
Validity matters because a worthwhile personality questionnaire should correlate with outcomes such as current job performance or even predict future performance and behaviour. Some better tests predict emotional intelligence, leadership behaviour, rate of promotion and burnout.
A personality questionnaire can be reliable but not valid. Reliability is necessary but insufficient for a useful test. For example, we could measure the resting blood pressure of employees – and we would expect it to stay more or less the same over different days.
Despite this reliability, though, it would not be a valid test of outcomes such as interpersonal skills or behaviour at work. In the same way, many questionnaires have good reliability data but low validity in terms of predicting people’s interpersonal skills or performance.
No test measures personality perfectly – they only measure it more or less accurately.
However, having good predictive validity separates more useful from less useful tests. Unfortunately, several well-known personality questionnaires have been criticised for their poor validity. For instance, the Myers-Briggs Type Indicator (MBTI) is described as being based on the theories of the psychoanalyst Carl Jung, but in 2018 researchers in Germany and the UK headed by Richard Bailey raised doubts. A separate team led by the University of Ottawa’s John Hunsley concluded that ‘the MBTI bears little correspondence to measures of vocational preferences and job performance’.
In contrast, substantial research evidence shows that personality is best assessed in terms of between five and seven dimensions. Perhaps the most researched modern classification measures extraversion, conscientiousness, openness to experience, emotional instability and agreeableness. Hundreds of studies show that these predict workplace outcomes such as job performance, aggression, delinquency and leadership potential.
Professor of psychology Adrian Furnham suggests that one reason for the MBTI’s continuing popularity is because it does not measure emotional instability, which allows its fans to say that the test makes no judgment about what is desirable versus undesirable – or right versus wrong.
Business school professors Randy Stein and Alexander Swan suggest that the MBTI is best considered ‘divorced from scientific process’. Some people do believe in so-called post-truth ideas that are not backed by robust data.
The results of even a robust personality test should be seen as hypotheses that a person has certain preferences and patterns of behaviour. A psychologist should ideally interview the individual to gather further evidence either to support or to refute the hypotheses generated by the questionnaire, mirroring the use of tests in medicine.
Personality tests can add substantial value when used to evaluate candidates for director and executive level roles; they can also help people to identify strengths and blind spots. However, be sure to seek advice on choosing tests that are based on rigorous data rather than mere anecdote and personal beliefs.
Dr Rob Yeung is an organisational psychologist at leadership consulting firm Talentspace.
CPD technical article
"Many questionnaires have good reliability data but low validity in terms of predicting interpersonal skills or performance"