April 2009 Issues of Merit

The U.S. Merit Systems Protection Board (MSPB) recently released their April 2009 Issues of Merit, with several interesting articles, including:

- Using Internet technology to recruit (public sector employers, are you on GovLoop?)

- Improving hires through multiple hurdles (e.g., accomplishment records)

- HR practices to increase employee engagement (supervisor selection, orientation, etc.)

- How to prepare a structured interview

Also, a very interesting note on the last page about OPM removing the time-in-grade requirement for promotions in competitive service!

Is your test the 40-yard dash?

A fascinating article came out in the last couple days about how the 40-yard dash is used to evaluate NFL prospects. And while I'm not a huge fan of equating sports teams with other organizations, sometimes the comparison works.

The article starts by describing the story of Brice McCain, a former defensive back from Utah. McCain was considered "too small" by football scouts until they came to town and saw him run the 40-yard dash; he did it in 4.30 and 4.34 seconds. As the article states, "Suddenly, his size (5-foot-9) was less of concern."

Consider these other points from the article:

- McCain's 40-yard dash time wasn't his only strength; he got high marks in other drills that assess quickness rather than overall speed (multiple-method convergence, anyone?)

- Scouts often feel you can teach things like catching, but teaching prospects to run faster is difficult (abilities vs. skills?)

- The 40-yard dash is considered more relevant for evaluating receivers and defensive backs than linemen, where scouts are looking more at foot speed and agility (job analysis informs assessment choice)

- Times for the dash vary with surface (grass v. artificial surface) and runners are never clocked wearing their uniform (beware fidelity of the test instrument)

- The importance of the test is debated given that few players ever run 40 yards during a game (some criterion-related validation might be in order)

- Businesses have sprouted up that provide physical training to prospects to help them perform better in front of scouts (test prep industry expands its reach)

There's quite a lot here that overlaps with assessment in general.

Where doesn't the comparison work? Well, non-sports organizations almost never have the wide variety of statistics available to them to use in judging applicants that sports scouts do. Many organizations also don't have recruiters constantly traveling around the country evaluating groups of applicants. And course there's that whole draft thing.

Still, an example of how assessment can be found in all kinds of situations, and how sticking to best practices pays off in a variety of situations.

The state of UIT

Many recruiters and assessment professionals believe that the future of employment testing lies with screening applicants over the Internet (rather than in person). And while there are substantial benefits associated with this method (e.g., convenience, speed), there are plenty of concerns as well (e.g., cheating, validity). Research in this area is in its infancy, which is why it is refreshing to see a full half of the March 2009 issue of Industrial and Organizational Psychology is devoted to describing the state of unproctored internet testing (UIT).

What struck me most about when reading these articles is the variety and excitement surrounding this field but even more, the tremendous lack of consensus in the professional community has about important issues related to UIT.

The articles start off with an updated summary by Nancy Tippins, who with her colleagues provided a heavily cited summary in 2006. This focal article is followed by twelve response articles and a final summary by Tippins.

Here's a (very) brief summary of some of the important points raised by the authors:

1. Cheating happens. But let's not forget that proctored tests have always been susceptible to some degree of cheating (e.g., via test question memorization).

2. There are many ways to mitigate the risks associated with cheating. This includes retesting, identity verification, and response pattern analysis. But it's not clear how successful these measures are, or even how needed they are.

3. Although there are potential legal risks (e.g., lack of standardized administration), UITs have not been directly evaluated in court.

4. The choice of whether or not to use UIT is influenced by many factors, not the least of which is the organizational reality communicated by upper management.

5. Some applicants may be turned off by an organization that uses assessments so obviously prone to cheating. But this may be balanced by increased convenience, speed, and immediate feedback.

6. Although cheating and response distortion occurs, it's unclear to what extent it impacts validity.

As an interesting note, the most common types of tests delivered via UIT seem to be biodata, personality, situational judgment, and T&E and preference questionnaires. There are also those who are administering cognitive ability tests in this way, sometimes adaptive.

UIT is in many ways the Wild West of employment testing. It's exciting and innovative, but comes with risk and lots of unanswered questions.

Here's hoping our field quickly speeds up the research side because this ship has clearly sailed.

HRE announces best HR ideas for 2009

Recently Human Resource Executive posted their Best HR Ideas for 2009 winners. Among the topics (benefits, HR tech, talent management, and T&D) they highlight some pretty interesting examples related to recruitment and assessment. Here's a sample:

- AlliedBarton Security Services gathered productivity data and discovered many managers were leaving at the peak of their productivity; this allowed them to focus additional resources on targeted retention efforts

- US Cellular uses web-based meeting technology to allow hiring supervisors to communicate about their candidates, ensuring more consistency and enhancing communication

- Proctor and Gamble, working with DDI, moved their cognitive ability testing online and made them adaptive

- SITA developed a leadership development portal where employees can literally "gauge" their competencies and receive targeted instruction

- BPO firm iQor created an online test designed to measure "emotional, intellectual, and conversational skills", reducing the need for interviews, increasing productivity, and reducing turnover

SIOP has a blog!

Well it's been a long time coming, but SIOP finally has a blog, the SIOP Exchange (with feed here).

Right now it's focused on updates from the conference, opening today in New Orleans, but moving forward it will broaden to "news and issues in the I-O community." Definitely a welcome addition to the blogosphere!

And speaking of the blogosphere, IPMA-HR's blog has a new location, along with a feed now. The most recent post is about a topic near and dear to me--social networking and performance management.