A review of situational judgment tests

In the latest issue of Personnel Review, Dr. Filip Lievens and colleagues provide an empirical review of situational judgment tests (SJTs), focusing on studies from 1990-2007.

SJTs, sometimes referred to as low fidelity simulations, present test takers with a scenario and ask them to select the appropriate response. Candidates may be asked to select what "should" they do, what "would" they do, the best response, the worst response, or some combination of the above. Here's an example:

You have been assigned lead responsibility for two weeks in the absence of your supervisor. On your first day in this role, one of your new direct reports comes into your office and complains that they were sexually harassed by the security guard when they entered the building. They ask that the situation be kept confidential. What would be your first action in response to this situation?

1. Contact the security guard and conduct an interview to obtain all the facts.
2. Assure the direct report you will look into the situation but cannot guarantee confidentiality.
3. Contact your supervisor to obtain instruction on next steps.
4. Conduct informal interviews with your other direct reports to determine if they have been harassed.

SJTs have some great benefits, and this article points them out. First, they can be valid predictors of performance--particularly when based on job analysis. Second, they show incremental validity beyond cognitive ability and personality tests, making them a valuable addition. Third, group differences tend to be reduced compared to ability tests, particularly when the cognitive load is low. Fourth, applicant perceptions of SJTs tend to be positive. And fifth, SJTs allow you to test large candidate groups simultaneously. I would add that they allow for all kinds of scoring possibilities as well (e.g., +1 for correct response, -1 for incorrect).

SJTs aren't without drawbacks--two major ones to be exact. The first is they can be susceptible to faking, practice, and coaching effects--although how they're built plays a large role in how big these effects are. The second is that we don't always know exactly what SJTs are measuring--is it job knowledge? Personality? Cognitive ability? The authors point out that more research is needed.

Overall, a very good review of a test method that every assessment professional should have in their tool belt. You can read an in press version here.