Evidence-based recruitment and assessment

Recently I've been reading Pfeffer and Sutton's Hard facts, dangerous half-truths and total nonsense: Profiting from evidence-based management.

It's one of those business books that's both entertaining and enlightening. As you know, there is a LOT of "fluff" out there--this is not one of those books. The authors are all about using good data and experimentation to discover what really works, not just what sounds good or what someone else recommended.

Case in point: the "War for Talent." This phrase gained popularity in the late 90's through several employees of McKinsey Consulting and their book of the same name. Those authors argued that the best performing companies had a deep commitment (obsession?) with finding and promoting talented individuals and offered data that claimed to support a link between this mindset and firm performance. But as Pfeffer and Sutton point out, a closer look at the data raises some eyebrows. Specifically, talent management practices were measured AFTER performance measures, resulting in a classic case of correlation-causation confusion.

Certainly Pfeffer and Sutton aren't the only ones to raise concerns about a talent obsession, but they do so in a very accessible and thorough manner. They highlight three poor decisions practices that apply to talent management (as well as many other issues):

- Casual benchmarking (for example, the failure of "Shuttle by United" to copy Southwest Airline's success or U.S. automotive companies attempting to copy Toyota's success). We see this in our field when folks want to know "how other people are recruiting" or "what test everyone else is using." Good information to know, but look before you leap.

- Doing what (seems to have) worked in the past (for example, using incentive pay in your new organization because it seems to have worked at another one). The best example of this is interview questions by managers who just know their questions about favorite books and who they'd want on a deserted island work--even though they don't have any data to support their view. In my experience about 20% of managers are good interviewers (and I place a lot of the blame on HR).

- Following deeply held yet unexamined ideologies (for example, equity incentives, the so-called "first-mover advantage", and merit pay for teachers). In our area this includes things like believing applicant tracking systems always result in improvement, or that integrity tests are more discriminatory than other types of tests.

So how do we apply these lessons to recruitment and assessment? Here are just a few ways:

1. Be a critical thinker. We know we're supposed to eye HR metrics with some skepticism, but do we? Do we adopt "best practices" without thinking about how our organization might differ in important ways? Are we lured by shiny new pieces of technology without asking ourselves whether we might be better off without it? On the flip side, do we resist new ways of doing things without even considering the possibilities?

2. Know the evidence. HR is not guess work--we know a lot about what works and what doesn't. Every HR practitioner and manager should read Rynes, Brown, and Colbert's "Seven common misperceptions about human resource practices", with a more detailed analysis here.

3. Push back when you hear something that sounds too simple or too good to be true--it probably is. Two examples: behavioral interviewing does not solve all of our assessment problems, and social networking sites will not solve all our recruiting problems.

4. Model evidence-based decision making. Make it clear that you are making decisions based on the best data you could find/gather and that this is an expectation for everyone. Rather than rushing into a decision, take the extra time to gather whatever information you get your hands on--as long as it doesn't lead to paralysis by analysis.

5. Do experiments whenever possible. Include an assessment instrument as a research-only tool and see if it predicts performance. Try out different advertising methods and mediums and track applicant numbers and quality. Did you know Yahoo! typically runs about 20 experiments at any time, changing things like colors and the location of text? We can't all be Yahoo!, but we can all be experimenters.

Some of my favorite quotes from the book...

"If doctors practiced medicine the way many companies practice management, there would be far more sick and dead patients, and many more doctors would be in jail."

"The fundamental problem is that few companies, in their urge to copy--an urge often stimulated by consultants who, much as bees spread pollen across flowers, take ideas from one place to the next--ever ask the basic question of why something might enhance performance."

"Instead of being interested in what is new, we ought to be interested in what is true."

"There is really only one way around this reluctance to confront the hard facts, and that is to consciously and systematically understand the psychological propensity to want to both deliver and hear good news and to actively work against it."