The Winter issue of Personnel Psychology showed up in my mailbox yesterday, and although there's only one article that directly addresses recruitment/assessment, it's a good one.
The study by McDaniel, Rothstein, and Whetzel analyzed whether publication bias exists in the validity information presented in technical manuals of 4 test vendors. To be honest, I was shocked that only 2 of the 4 showed evidence of moderate-to-severe bias. In both cases, the publisher tended to report only statistically significant correlations.
Unfortunately the names of the test vendors were not given; they were specifically omitted to avoid stigmatizing the vendors and to make the article easier to review. IMHO both of these reasons are weak, but that's neither here nor there. Still, a great introduction to the "trim and fill" method and provocative results that have implications for any meta-analysis.
Another reason this article is interesting is it references a chapter written in 2005 by Sue Duval where, using the same "trim and fill" method, the criterion-related validity reported by McDaniel et al. in 1994 for structured interviews (.27) was found to be likely be an overestimate, and the true value (.21) is closer to that of unstructured interviews. As McDaniel himself points out in an upcoming publication, this is not the last word on the validity of structured interviews, but it certainly gives one pause! That study, combined with the 2005 study by Roth, Bobko, and McFarland on work sample tests really cry out for a re-do of Hunter & Schmidt's seminal paper.
By the way, big props to Dr. McDaniel for allowing access to so many of his publications.
No comments:
Post a Comment