Monday, March 10, 2008

Warning: Warnings may not work

Although faking is a persistent issue in personality testing, no one agrees on the best way to handle it. Some have suggested including "warning" statements in the test: letting applicants know there is a lie scale or some repercussions for false responses. But researchers are far from agreed on this strategy.

Now a new study out in the latest issue of Human Performance adds weight to the argument that warnings may not help us avoid the faking issue.

In the study, researchers had 464 participants fill out personality inventories in either a "warned" or "unwarned" condition. They then looked at the convergence of their scores with scores given to them be observers.

Results? Lower mean scores on some personality dimensions (which is often what happens) but no improvement in the convergence between self- and other-ratings. So in other words, it made a difference, but not a significant one. Implication: simply warning applicants that there are consequences for "inflating" their scores may not do much. Fortunately, it may not matter as well-constructed personality inventories (when used properly) still show useful levels of validity.

The other article in this issue related to assessment looked at the relationship between personality (specifically neuroticism), self-efficacy, gender, and performance (alternate version here). Using data from nearly 900 freshman from 10 different U.S. colleges and universities, the author found several results:

- female participants reported significantly lower levels of emotional stability and (to a lesser extent) self-efficacy--an important consideration if using these scores for selection

- there was a positive relationship between emotional stability and self-efficacy for female participants but the relationship for males was "nearly zero"

- emotional stability and gender interacted to affect self-efficacy which, in turn, affected performance (measured by GPA)

The last point is (to me) the most interesting, as it suggests that personality scores may predict performance indirectly through their relationship with other constructs (in this case, self-efficacy). This suggests another layer of analysis is needed when looking at the utility of personality tests.

No comments: