Upcoming webinar on defending your tests

Tests aren't valid or invalid per se--it depends on what you use them for.

But if your tests are challenged legally (say, because they have a discriminatory impact against a protected group), one of the things you'll want to defend yourself with is a test validation report--a documentation of why the test was developed, how it was developed, and the purposes for which the test should be used.

This is one of the topics that will be covered in an upcoming webinar sponsored by Talent Management and presented by some well known folks over at APT. Taking place on November 11th at 11am PST, the webinar is titled "Testing the Test: What You Need to Know about Test Validation, Litigation, and Risk Management."

Should be worth a watch/listen.

On a related note, the latest issue of Talent Management had some good articles in it, including ones on "role based" assessment (which just sounds like good 'ol fashioned position-based assessment) and employee surveys. It's actually not a bad little magazine, and it's free. You can subscribe here.

Leadership podcast

Seems to be the season to discuss leadership. Wonder why?

A recent guest on the Doug "lawyer to peacemaker" Noll show was Dr. Robert Hogan, renowned expert on leadership and personality and President of Hogan Assessment Systems. Dr. Hogan discussed several things, including:

- how evolution relates to leadership and followership

- why the current trend of focusing on "strengths" may be misguided

- the base rate of bad leaders (hint: it's not 5%)

- what followers want in a leader (hint: it's not a big ego)

Good stuff. Reminds me that I need to join 2002 and start doing more audio/video.

Discrimination, assessment centers, and handshakes

The title sounds like a strange combination, no? That's because it refers to three separate pieces of research published in the September 2008 Journal of Applied Psychology.

First, Umphress et al. describe a study that demonstrates how important leaders can be in setting the tone for selection. Specifically, the authors found that when authority figures focused team selection decisions on job performance factors, individuals that had a tendency to discriminate based on social dominance orientation were less likely to do so. Implication? To help avoid discriminatory hiring and promotion decisions, focus decision makers on job-related performance factors. That way, they're less likely to rely on their own biases.

Next, Meriac et al. with a meta-analysis of the incremental validity of assessment center (AC) ratings over other assessment tools. Specifically, the authors found that AC ratings explained a "sizable proportion of variance in job performance" beyond cognitive ability and personality tests. Good news for fans of assessment centers out there.

Last but not least, Stewart, et al. describe the results of a study of 98 undergraduate students that participated in mock interviews. In the words of the authors, "quality of handshake was related to hiring recommendations." How exactly does that work? Apparently how you shake hands sends messages about your degree of extraversion (above and beyond your appearance). The authors also found that the effect seemed to be stronger for women than men. Implication? For those of you interviewing for sales jobs, pay attention to your handshake!

Honorable mentions:

- Judge & Livingston on how traditional gender role orientation impacts the wage gap

- Cascio & Aguinis on trends in I/O psychology from 1963 - 2007 (good stuff; read here)

- Chiaburu & Harrison's meta-analysis on how co-workers impact job performance

- Levi & Fried on differences between African Americans and Whites on attitudes toward affirmative action programs

FAA uses games to hire and train

Turns out all I have to do is post about how we should be using video games for recruitment and assessment, and an example appears!

In this recent article in the New York Times (should be first link), the author describes how the Federal Aviation Administration (FAA) is using a sophisticated simulator to train air traffic controllers.

Motivated primarily by the impending retirement wave and massive need for new controllers (1,700 a year for the next 10 years) , the FAA has developed a multi-screen simulator that allows trainees to hone their skills in a safe but semi-realistic environment. From the article:

"The tower simulation is realistic. Aircraft first appear as tiny dots against blue sky, clouds or stars. On the ground, drivers of maintenance trucks ask permission to cross a runway so they can fix a lighted sign. A click of the instructor’s mouse can shift the time of day, and change the weather — from rain to hail or cloudy to clear. To make the simulations as unpredictable as in the real world, some pilots ignore instructions."

But it's not only the training that's innovative. The FAA's screening process puts most of ours to shame. Specifically, the candidates complete a six hour (which might be overkill) computerized aptitude test that measures geometry and math ability.

This is followed by "game-like tests" designed to measure things like ability to work under pressure, maintain "situational awareness", short- and long-term memory, multitasking, and flexibility. The tests vary from air traffic simulations to ones that look like Frogger or Tetris.

So someone out there gets it! The system is even decribed as "a big Xbox."

But they could do even better. Here are some ideas how:

1. Use the multi-screen simulator for recruitment and selection, not just training. I really hope they show the system off during recruitment open houses--I know I would. And if it isn't cost prohibitive, and makes sense given entry level requirements, why not use the simulator as part of the screening process?

2. They've got a pretty good recruitment site and make good use of video. Why not add a java-powered mini-game that simulates the job? Maybe have a leader board and allow people to put in their e-mail and opt-in to getting more information about becoming an air traffic controller?

3. On a related note, why not go the route of America's Army and mix in a little SimCity and Flight Simulator and produce a more full-featured game that simulates the job? Again, players could have the option of uploading their scores to a public website, and allow them to enter their email (securely) to get more info?

Not all of our jobs lends themselves so well to simulations and video (I'm not sure SimHRManager would be very popular). But whenever possible, let's take advantage of the technology around us!

Googling applicants, RJPs, and engagement

The September 2008 Issues of Merit was just released by the U.S. Merit Systems Protection Board and has some articles worth a quick read:

- The drawbacks inherent in doing internet searches of potential applicants (e.g., finding inaccurate or misleading information, gathering information that could be potentially discriminatory)

- Realistic job previews (of which I'm a huge fan)--these can take the form of videos, classes (on-line or otherwise), or simply a better description of the job. MSPB gives the example that they provide applicants with a list of "This job might be for you if..." factors along side "This job might not be for you if..."

I've found "willingness" or "pre-screening" questionnaires to be helpful as well, which simply have candidates answer a series of questions related to the screening process (e.g, "Are you willing to have your credit reports checked?") or the job itself (e.g., "Are you willing to come in contact with toxic chemicals on a daily basis?")

- Performance management and employee engagement. MSPB studies have shown that the PM process itself is more important than the formal structure--so what you say to your employees is more critical to engaging them than whether they receive a report each year. Makes intuitive sense, but many organizations assume that because everyone receives an annual appraisal, their performance management system must be working!

What the new ADA law means for recruitment and assessment

On September 25, 2008, President Bush signed the Americans with Disabilities Act Amendments Act (ADAAA) or Senate Bill 3406 (actual text of which can be found here).

The changes the ADAAA entails have been well covered, especially over at George's Employment Blawg. The purpose of this post isn't to give a detailed analysis of the changes, but rather highlight what it means for recruiting and hiring.

First, a brief reminder of who is covered by the ADA. It covers not only those who have a substantially disability that limits a major life activity, but those who have a record of such a disability, and those who are REGARDED as having a disability (more about this in a second). Check out this information from the EEOC for a brief overview.

Given that introduction, there are two big things we all need to be aware of when it comes to the new bill:

1) Who is considered "disabled" under the ADA has just been increased dramatically. The ADAAA explicitly rejects several U.S. Supreme Court cases made in the last 10 years that narrowed who was considered disabled, and restores the broad scope. One big way they did this was by including those whose disability can be mitigated (e.g., controlling diabetes).

2) Individuals who claim injury because they were "regarded" as having a disability have a much easier time qualifying under that category.

So what does this mean? It doesn't fundamentally change anything we do when recruiting or assessing candidates. But it does mean this: recruiters and hiring managers have to be even more careful about making assumptions when sizing up candidates. Because more people are now covered by the ADA, our potential risk has expanded.

And because of the expansion of who is covered, particularly in the "regarded" category, recruiters and hiring managers should receive explicit training not to assume someone is disabled and therefore can't do the job. Their inquiry (if they must make one) should be limited to this question: "Can you perform the essential functions of the job, with or without reasonable accommodation?"

That's the only question they should ask on this topic, and they should ask it of all candidates. Of course they should also be prepared to answer this question: "Well I'm not sure. What are the essential functions?" (another great use for job analysis)

So bottom line: big change to the law, relatively small change to the way we do business. But a good reminder of our responsibilities.

p.s. other good reads regarding ADAAA include this article and this essay