Test validity: A fairy tale

Once upon a time there was a field called Industrial and Organizational Psychology.  Its researchers and practitioners dealt with a myriad of magical issues ranging from individual differences and behavior to organizational structures.

Within this field, there was a specialty called Personnel Psychology.  It dealt with narrower--but no less mysterious--issues such as defining and designing jobs and, most relevant for us, finding and hiring the right people.  The I/O psychologists and HR professionals that quested for these answers often found themselves on dangerous missions like battling Monsters of Doubt (i.e., first-line supervisors).

These adventurers had two main weapons at their disposal when fighting these monsters: the Carrot of Truth and the Stick of Pain.

When invoked, the Carrot of Truth, fashioned deep in the Mines of Correlation, caused monsters to realize that hiring the right people was the right thing to do for their realm.  It increased productivity and morale, customer satisfaction, and organizational flexibility.  It also allowed supervisors to spend more time leading and less time dealing with gremlins (i.e., employees with performance problems).

The Stick of Pain used an opposing form of magic but was sometimes equally effective.  It attempted to slay the monsters using a peculiar power called The Law.  The Law frightened monsters because it meant they could experience emotional pain and suffering, and--more importantly--fewer bags of gold.

For a long time, Personnel adventurers used both of these weapons to slay all kinds of monsters, on high mountains and in dungeons.  But over time, the adventurers discovered something: the Stick of Pain was becoming less and less effective.

It wasn't that the Stick was powerless.  It's just that its magic didn't seem to frighten the monsters as much.  The monsters saw their gold piling up and didn't feel the sting of suffering as they once had.  And they started developing an addiction to carrots all on their own.

So there came a day where the adventurers and the monsters met on the battlefield and came to an agreement.  No longer would the adventurer wield the Stick of Pain.  And in return, the monsters pledged to respect the Carrot of Truth.  They forged an eternal partnership and lived happily ever after.

The End.


Okay, so I've taken a little artistic license with my blog post today.  But hopefully you see where I'm going.

Back in the old days (ya know, like the 80s), employers were faced with a foreboding world of testing, with the Civil Rights Act and cases like Griggs vs. Duke Power looming large over their assessment programs.  I/O psychologists were brought in to help organizations navigate the complicated world of employment testing, which required an appreciation of statistics and the law alike.  Large awards and settlements brought C-level attention, and regulatory agencies like the EEOC and DOJ served in ongoing oversight roles, requiring employers to clean up their act with procedural requirements that could be burdensome.

Nowadays, I/O psychologists are as likely to be valued for their ability to crunch "big data" to detect employee behavior trends as their ability to conduct thorough job analyses (not that the two are mutually exclusive).  Lawsuits regarding testing are infrequent compared to issues like wages and hours, harassment, and terminations.  The selection cases that do come up are as likely to involve disabilities as adverse impact due to cognitive loading.

Sure, we have the occasional big case that gets attention.  But the bottom line is over the years the "stick" has become much less effective as an argument for sound assessment than the "carrot."

Smart employers like Google have started crunching the numbers and realized the true business value of defining the right competencies for jobs.  They're doing so not because they're afraid of litigation, but because they see more clearly the direct line between best practices in selection that we've been preaching for years--i.e., focusing on valid assessment results--and the bottom line.


So where does that leave the stick (i.e., fear of lawsuits)?  Is it time to put it away along with phrenology and T&Es (woops, that slipped in)?

I don't think so.  Organizations will always be subject to legal scrutiny when their selection processes have adverse impact and the right person talks to the right attorney.  Personnel psychologists and HR professionals should always have a healthy respect for the legal climate we operate in, and not forget that "job related and consistent with business necessity" isn't fictional gibberish.

But what it does mean is that because organizations are paying attention to their assessments, they are more likely to yield valid results and be more free of illegal bias.  That means management's quest and the selection professional's quest are more likely to converge, with a lot more cooperation.

And hopefully a lot fewer monsters.