One way to reduce interviewer leniency/severity

A persistent challenge in interviews is that certain interviewers tend to be lenient (i.e., score candidates highly) while others are consistently more severe (i.e., score candidates lower).  This of course is not ideal as it introduces measurement bias as well as reduces the defensibly of the process.

One way to reduce these tendencies discussed by Hartwell and Campion in the June 2016 issue of Journal of Applied Psychology is to provide interviewers with what they call "normative feedback interventions."  Basically what this means is giving interviewers data on how they rate candidates over time compared to how other interviewers rated.  It can reveal to interviewers that they tend to rate candidates more harshly, or more easily, than others.

What Hartwell and Campion found in their study (of over 20,000 interviews using more than 100 interviewers) is that by providing this feedback to interviewers, it minimized interviewer differences and increased interview reliability--both obviously good things in terms of quality of the process.  Interestingly, it did not seem to impact the validity of the interviews, but it did impact which particular candidates were hired.

Up until now, one of the most often recommended practices for reducing rating errors has been pre-interview instructions and guidance regarding these errors.  What this study suggests is we can do even better by providing interviewers with objective data about their ratings over time.  Listening to someone talking about rating bias probably feels a lot different than actually seeing how you do compared to your peers!

Some easy tests to improve your hiring success

The interview is such a commonly used hiring assessment that it's hardly worth mentioning (although there is always room for improvement).

But what if you're already doing interviews and you want some easy to implement add-ons?  No problem.  Here are some ways to improve quality of hire for knowledge worker positions that don't take a long time, an automated solution, or a PhD to develop:

1. Pre-screening questionnaire.  Whether you use something quick and cheap like SurveyMonkey or your own proprietary assessment system, it's easy to create open- and closed-ended items that serve to screen out the uninterested, allow you to get some more detail from candidates, and even help you solve problems you've been struggling with!  Keep it relatively short so you don't dissuade the most in-demand candidates.


2. Targeted cover letter.  Don't just ask for a generic cover letter, ask applicants to describe in their letter how their background syncs with the core competencies you're looking for.  Remember: limit the length; two pages is generally sufficient.



3. Research project.  As part of the application process, as candidates to look into an issue that's relevant for the job.  How do they think the new overtime regulations will impact the industry?  What new technologies are on the horizon that will change the way this job is done?  Have them briefly write up their results, and/or ask about it during your interview.

4. Writing exercise.  There's no substitute for live demonstrations of writing ability.  Have them correct a document you've messed up, ask them to write a quick memo to a customer--just something related to the core duties of the job that you would expect them to be able to do day one.

5. Rule/procedure application.  Knowledge worker jobs are characterized by frequent application of laws, rules, and procedures to specific situations.  Either provide them with the rules ahead of time or give them the short version, then give them a specific fact pattern and have them come up with a solution or options.



6. Oral presentation.  Have you ever been on a hiring panel where there's an oral presentation?  If not, you're missing out.  Presentations are a great way to mix things up and see those different skillsets that a candidate brings.  Of course realize that you're adding a presentation to an interview, which I'm pretty sure are both in the top five of most stressful events.


Notice that you can mix and match theses approaches: have them do a rule/procedure application and then write a memo.  Have them do an oral presentation on a topic they researched ahead of time.  Of course, the tests you use should be based on the requirements of the job; start with entry level KSAs needed and let the assessments flow from that.  Beyond that, be creative!