Living with T&Es

There are a number of ways to use the Internet to perform personnel assessment. Examples include timed cognitive and job knowledge tests, biodata instruments, and personality inventories. But one of the easiest--and thus most tempting to use--types of tests is known as training and experience questionnaires, or T&Es (also sometimes called E&E for education and experience).

A typical (poor) example of a T&E item might be something like this:

How much experience do you have conducing job analysis?

a) None
b) Less than 2 years
c) 2-4 years
d) 4 or more years

I could go on at length about the challenges inherent with using this type of assessment, but I'll spare you. Instead I'll point you to Jim Higgins' December 2008 newsletter, HR Rampage, in which he addresses this topic, among others (see page 2).

Jim points out several problems with this type of assessment, including the overwhelming urge to self-inflate in high-stakes scenarios (we already have problems with outright cheating), the inability of highly qualified individuals to give themselves sufficient credit, and the work required to validate responses.

So given all these challenges, what can we do to mitigate them? The solutions investigated so far (e.g., elaboration, warnings) have met with very limited success. But there are a number of tactics we can take in this situation. Here are some other methods to consider as we wait for more research in this area:

1) Accurately describe the job and requirements to prevent an unqualified individual from applying in the first place.

2) Clearly word stems and responses to avoid legitimate mis-reads.

3) Include lie items (e.g., "I have experience using the HR Tests Job Analysis Technique") and deduct points when candidates endorse them.

4) Use false bottoms (e.g., both (a) and (b) are worth zero points) and false tops (e.g., both (c) and (d) are worth the same).

5) Use scales appropriate to the item. For example, amount of experience is often the incorrect scale; type of experience is better.

6) Encourage hiring supervisors to follow up on specific items in their interview.

7) Use a friendly zero point, such as "I do not have any experience but I would be willing to learn" instead of "none." Remember there are egos involved here.

8) Ask questions that are appropriate for a T&E. Don't ask candidates to rate their oral communication skills.

9) Don't ask people to compare themselves to others (e.g., Average, Above Average). Instead use objective measures such as number of times.

10) Seriously consider weighting the items. This is of course dependent upon subject matter expert input, but it's highly likely that your SMEs consider certain training or experience areas more important than others.

11) Before they even get to the T&E, use willingness/pre-screening questionnaires that ask candidates to acknowledge they understand the less-than-perfect conditions related to the job (e.g., mandatory overtime, working outside in the heat).

12) Consider using them as a feedback tool for candidates rather than a scored instrument (e.g., "Your responses indicate you have very little experience and education related to this job. Would you like to continue to apply?").

13) Base everything on SME input. Yes, I realize this probably doesn't need to be stated, but one of the worst temptations is for HR folk to draft T&Es themselves thinking they're easy to write. This is a myth, and helps contribute to poor quality eligible lists.

This is by no means an exhaustive list, and I'm sure there are other methods out there for helping us live with T&Es. Feel free to add your suggestions!