Research update

Several research updates this time, including fascinating studies of how computers can assist with interview training, and how a brief writing exercise can lower stereotype threat for women:


International Journal of Selection and Assessment, December, 2016:

- International support for the cultural intelligence scale

- Looking to improve applicant interview performance? Maybe a computer can help.

- This study found that time lag and g-loading are important factors impacting re-testing results

- Status-seeking seems to be an important individual difference when looking at self-presentation behaviors, including exaggeration and faking in job search

- Development and validation of a 360-degree measure of leadership personality


Personnel Psychology, Winter 2016:

- Do CEOs significantly impact firm performance?  This study found evidence that they do.

- More evidence that the assumption that performance is normally distributed is questionable

- A more accurate correction for range restriction is presented, and an example analysis indicates the relationship between the Big 5 and job satisfaction may be greater than previously believed



Journal of Organizational Behavior, November, 2016 (which includes several articles devoted to the importance of theory in organizational sciences):

- Are self-focused or other-focused recruiting advertisements more effective?  This study suggests it depends not only on cultural differences but individual regulatory focus



Journal of Applied Psychology, October, 2016:

- A suggestion for improving meta-analytic structural equation modeling



Journal of Applied Psychology, November, 2016:

- A fascinating study of how having women compose a brief written description of their personal values can help ameliorate stereotype threat in competitive environments

How to create the ultimate hiring system

Unless your organization is primarily composed of robots (which is becoming more of a reality for some*), arguably the most important thing you need to get right is hiring.  To restate the obvious, without the right people, in the right places, at the right time, your organization hampers its ability to innovate, collaborate, deliver, and fulfill your mission.

So how do you ensure that your organization consistently makes great hires?  Before I get into the steps, let's talk a little bit about culture.  None of the steps below will reliably deliver results unless you first get serious about two things: (1) disciplined processes and procedures, and (2) a clear understanding of the roles of HR versus hiring supervisors.

In order to make great hires time and time again, you have to document your practices and put them in place across the organization.  Everyone needs to understand that this is the way things are done here, not simply an initiative.  Supervisors and HR need to be trained--and reinforced--for how successful they implement these steps.

Speaking of these players, both hiring supervisors and HR need to be very clear about what their roles are.  Hiring supervisors are expected to know the job they're hiring for and what it takes to succeed in it.  HR is expected to have expertise in job analysis, recruiting, assessment, onboarding, and other aspects of talent management.  This should be part of their job descriptions, and their performance should be in part based on how successful they are at serving in these roles.

Okay, with that out of the way, let's talk about the steps your organization needs to have in place to ensure repeatable success in hiring.

Step 1: Know your Organizational Reputation.  Before you even think about hiring for a specific job, you need to think about the reputation of your organization.  Is it a destination employer, or an employer of last resort?  What do people say about your workplace?  This is important because it drives the pipeline of talent.  If you're a destination, the pump is primed and "hard to recruit jobs" becomes less of an issue, making the steps below that much easier.  Find out what the word on the street is about your organization--how do your employees feel? Your customers? Prospective applicants?

Step 2. Analyze the job.  Yes, many jobs are becoming more fluid, but even narrowing the job to its occupational category helps.  Think about the most important tasks the person will perform on a day-to-day basis and what competencies or knowledge, skills, and abilities are required to perform them.  Sites like O*NET are a huge resource so you don't reinvent the wheel.  Without knowing the job, hiring is a roll of the dice.

Step 3. Develop a recruitment/assessment strategy.  Ya gotta have a plan.  It doesn't have to be a 20-page missive, but you need to document what your plan is, otherwise you're unlikely to cover all the bases.  Honestly here's where a lot of hiring processes fall apart--people have the best intentions but they forget about certain key steps.  Hey, here's an idea: use the same document that you used to document the job in Step 2!  The way the key competencies will be linked to how you plan to recruit and assess for each of them.

Step 4. Use multiple and creative recruitment strategies.  The only time "post and pray" is acceptable (and even then I'd argue against it) is if you've nailed your reputation as described in Step 1.  Recruiting is sales, plain and simple--you're selling the job, the organization, and the people.  Use the web, but also think about physical interactions, including open houses.  Reach out to schools.  Develop realistic job previews.  Hire recruiters that have a marketing and sales background.  Don't be afraid to push the envelope if you need to stand out from the crowd.  Honestly, the sky is the limit.

Step 5. Use multiple high-quality assessments, internet and mobile whenever possible.  In-person interviews aren't going away any time soon (although many of those are migrating online), but they should be only one tool  in your belt--not the only tool.  Assessment starts with how you recruit, because you allow applicants to self-select in and out.  It continues with assessments embedded in the application process, whether that's a statement of qualifications, an online survey, or a set of online skills assessments.  And don't forget to make any "minimum qualifications" truly minimum--please don't rely on hard-and-fast "X years of experience" or "Y degree"--those should be suggestions.  The important thing isn't the type of assessment, it's that you're using several and they're tied to those important competencies you identified in Step 2.  In short, using a single assessment is like buying a house based on what it looks like from the outside.

Step 6: Don't forget about them once you make the offer.  Again, this is pretty obvious, but once you've made the offer, don't breath a sigh of relief and get back to your Inbox--your job isn't over yet, not by a long shot.  Your new employee needs to feel welcomed to the organization, have the tools they need, understand what the expectations are, and get continual feedback--in other words, feel like they made the right choice and have a successful future with you.


Get these simple steps right, make them part of your organizational DNA, and you will ensure that not only do you get hiring right--you'll get performance right.

* Stay tuned for my signature article of 2018, "How to hire the right robot for the job"

p.s. this post marks 10 years for my blog!  Thanks for reading!

Research update

Several recent journals to pore through:

July 2016 issue of Journal of Organizational Behavior

July 2016 issue of Journal Applied Psychology, including:

- Computer scoring of candidate essays

- Everything you've been told about assessment center ratings is confounded

and the biggie...

September 2016 issue of International Journal of Selection and Assessment, including:

- Faking on assessment measures

- Emotional intelligence and service jobs

- A self-report measure of psychopathy in the workplace

Welcome to Sacramento, IPAC'ers!

This year, IPAC's annual conference is here in my home of Sacramento from July 31 - August 3.  I'll be there Monday afternoon for a session titled "Fits and Starts: The Evolution of Testing for the State of California (Special Invited Session)", where I'll be interviewing my good friend, Adria Jenkins-Jones.  Here's the description:

In this lively discussion, the presenters will discuss the current state of employment testing for the state of California, including significant recent and upcoming changes to our examination software. Using an interview format, the presenters will discuss the massive changes envisioned for statewide testing, and how the California Department of Human Resources is attempting to collaborate with stakeholders to change the traditional paradigm, and significantly improve automation, while maintaining the commitment to merit. The presenters will engage in open dialogue and there will be time for audience members to ask questions about current and future directions of testing for the state.

Feel free to use the comments section below to post the sessions you plan on attending, or anything about your experience!

Hiring for goodwill (not Goodwill)

Those of us that write, teach, and consult on personnel assessment usually paint the process as a very rational process broken into several steps:

Step 1) Study the job to identify the critical tasks

Step 2) Identify the knowledge, skills, and abilities (or competencies) required day one to perform those tasks

Step 3) Create selection systems that accurately measure candidates' levels of #2

Step 4) Hire the person(s) who demonstrate the highest scores on #3

But as you know, hiring is rarely so logical.  In some cases this is due to organizations not having the expertise, or making the time, to follow the steps above.

But in other cases, there are simply other factors at play--factors that can't be ignored by the decision makers because they exert such a significant influence on the process.  And these factors are largely intangible, meaning they're more difficult to quantify and therefore not typically included explicitly in a standard assessment process.

For the purposes of discussion, I'm not talking about negative factors, such as illegal discrimination or selections based purely on politics.  Instead, let's think of potentially useful constructs.

For example:

1) Physical presence related to professionalism.  Let's be honest, looks matter when it comes to interviews.  In general, polished shoes beat flip flops.  Suits beat sweats.  No scent beats heavy cologne/perfume.  But this may not be given a formal rating during the process--for several reasons, including the difficulty of rating and the potential overlap with discriminatory considerations.  But again, that doesn't mean it's not a factor.

2) Honesty.  How many times have you been on an interview panel and perceived a candidate more positively because he/she was honest--about a weakness, for example, or about what they need to be successful in the position.  It matters, and one reason it matters is because so many candidates hide behind a veneer of perfection.  But are you giving them a formal rating on honesty?

3) Potential.  The applicant may not have demonstrated the KSAs that are key for the job, and thus may score poorly on assessments that measure past performance.  But that doesn't mean they aren't capable of learning those KSAs, and even exceeding the performance of those that have already mastered them.  Why?  Because individual performance is more than just the KSAs someone brings to the job--it's also about the organizational environment, onboarding, development opportunities, the supervisor, and other factors that often go unmeasured during the hiring process (e.g., personality).

4) Preexisting relationships.  It's quite common for organizations to seek out individuals who have existing relationships that can be capitalized on.  For example, someone may have relationships with potential customers, or may be able to access a market in a different way.  If an organization is trying to enhance its relationship with certain key stakeholders, they may seek out someone who already has established connections.  Sales professions immediately comes to mind, but other types, such as consultants or leadership positions, may also benefit greatly from these connections.

In many ways these factors can be thought of as individual goodwill, similar to the accounting principle.  In this context, an organization's brand is an example of an intangible asset that nonetheless has great value.  And in a similar way, individual goodwill is difficult to quantify.

So what is someone that cares deeply about hiring right to do about all this?

First of all, acknowledge that these factors play a role.  Any attempts to link your assessments to outcomes that don't take these factors into account is missing a huge potential explanatory factor.  You may draw conclusions regarding your assessments that are simply false because of the nature of the selection decision.

Second, attempt whenever possible to build these intangibles into the hiring process.  For example, preexisting relationships could be considered part of a communication competency.  If hiring for potential, break down what the core abilities are that you're considering for potential--chances are you can measure those.  There are both overt and covert measures of honesty and integrity--that work.

Third, as an assessment community, we should all recognize that these factors play a role.  When we research, teach, and consult on hiring, acknowledge these factors, and help others understand how to take them into consideration when designing a successful selection system.

In the end, it's a relatively simple prescription: let's make the intangible, tangible.

One way to reduce interviewer leniency/severity

A persistent challenge in interviews is that certain interviewers tend to be lenient (i.e., score candidates highly) while others are consistently more severe (i.e., score candidates lower).  This of course is not ideal as it introduces measurement bias as well as reduces the defensibly of the process.

One way to reduce these tendencies discussed by Hartwell and Campion in the June 2016 issue of Journal of Applied Psychology is to provide interviewers with what they call "normative feedback interventions."  Basically what this means is giving interviewers data on how they rate candidates over time compared to how other interviewers rated.  It can reveal to interviewers that they tend to rate candidates more harshly, or more easily, than others.

What Hartwell and Campion found in their study (of over 20,000 interviews using more than 100 interviewers) is that by providing this feedback to interviewers, it minimized interviewer differences and increased interview reliability--both obviously good things in terms of quality of the process.  Interestingly, it did not seem to impact the validity of the interviews, but it did impact which particular candidates were hired.

Up until now, one of the most often recommended practices for reducing rating errors has been pre-interview instructions and guidance regarding these errors.  What this study suggests is we can do even better by providing interviewers with objective data about their ratings over time.  Listening to someone talking about rating bias probably feels a lot different than actually seeing how you do compared to your peers!

Some easy tests to improve your hiring success

The interview is such a commonly used hiring assessment that it's hardly worth mentioning (although there is always room for improvement).

But what if you're already doing interviews and you want some easy to implement add-ons?  No problem.  Here are some ways to improve quality of hire for knowledge worker positions that don't take a long time, an automated solution, or a PhD to develop:

1. Pre-screening questionnaire.  Whether you use something quick and cheap like SurveyMonkey or your own proprietary assessment system, it's easy to create open- and closed-ended items that serve to screen out the uninterested, allow you to get some more detail from candidates, and even help you solve problems you've been struggling with!  Keep it relatively short so you don't dissuade the most in-demand candidates.


2. Targeted cover letter.  Don't just ask for a generic cover letter, ask applicants to describe in their letter how their background syncs with the core competencies you're looking for.  Remember: limit the length; two pages is generally sufficient.



3. Research project.  As part of the application process, as candidates to look into an issue that's relevant for the job.  How do they think the new overtime regulations will impact the industry?  What new technologies are on the horizon that will change the way this job is done?  Have them briefly write up their results, and/or ask about it during your interview.

4. Writing exercise.  There's no substitute for live demonstrations of writing ability.  Have them correct a document you've messed up, ask them to write a quick memo to a customer--just something related to the core duties of the job that you would expect them to be able to do day one.

5. Rule/procedure application.  Knowledge worker jobs are characterized by frequent application of laws, rules, and procedures to specific situations.  Either provide them with the rules ahead of time or give them the short version, then give them a specific fact pattern and have them come up with a solution or options.



6. Oral presentation.  Have you ever been on a hiring panel where there's an oral presentation?  If not, you're missing out.  Presentations are a great way to mix things up and see those different skillsets that a candidate brings.  Of course realize that you're adding a presentation to an interview, which I'm pretty sure are both in the top five of most stressful events.


Notice that you can mix and match theses approaches: have them do a rule/procedure application and then write a memo.  Have them do an oral presentation on a topic they researched ahead of time.  Of course, the tests you use should be based on the requirements of the job; start with entry level KSAs needed and let the assessments flow from that.  Beyond that, be creative!

First, get the people basics right

Competencies, talent, gamification... there's no doubt about it, we like us some buzzwords.  Like bright shiny objects, these ideas entice--and largely detract.

Sometimes new ideas and ways of thinking can lead to significant improvements in the way organizations manage their people. But here's the truth that no one seems to want to talk about: many organizations fail to get the basics right.  So while leaders may be leaping headlong into the nanofied virtual talent management sunset, the foundation of HR is lacking.

What are these basics of which I speak?

1. Adequately defining jobs--based on subject matter expert data.   Every single job should be defined and documented in terms of key tasks, requirements, and expectations.  The form this takes is less important than the quality of the data. This is the bedrock that helps you recruit, select, reward, and manage effectively.

2. Recruiting like you're selling, not like you're being forced to.   Writing attractive job ads is so easy, why aren't we swimming in them?  The same reason many organizations fail to accurately describe the job: laziness and lack of discipline.

3. Using valid hiring measures.  Speed of hire is important, but not even remotely as important as quality.   I can make you a sandwich really quickly if it's just bread.   Do you think Google gets millions of resumes each year because candidates are hoping for a quick hire?  Importantly, the higher in the organization, the more time should be spent on valid assessment.

4. Holding leaders accountable for being leaders. This really should be #1 except I was trying to go chronologically (and will fail miserably).  All too often, it's the line staff who are quickly called to the carpet when they make mistakes.   But holding leaders accountable for their behavior (hint: ask their subordinates) is exponentially more powerful.

5.  Listening to each other.  Many if not most good ideas for improving your organization are in the heads of your line staff.   Do you ask them regularly and implement their ideas?  Is listening skill considered critical for all employees?

6.  Saying thank you.  It's easy, it's cheap.   Do it more, and mean it.  

7.  Dealing firmly with poor performance.  This is top to bottom, from not being helpful on the phone to running productive meetings.  Again, the higher in the organization, the more important this is.

8.  Growing your people--forever.  Sure,  they may leave, but they'll leave sooner if you don't invest in them.  And like everything else on this list, it grows your reputation. 

9. Treating people with respect and fundamental human decency.   If you have this as a backbone, many other things simply follow.  There's a reason why one of the most popular business books recently is The No Asshole Rule.

None of this is incredibly difficult, it just takes the most precious resource of any organization: time.   And it takes commitment and discipline.  But these aren't initiatives.   They're part of an organization's DNA--or not.  They're how people respond when asked what it's like to work there.   And who is responsible for ensuring they happen ?  The people at the top. 

So before your organization jumps on to the latest buzzword bandwagon, make sure it's getting these basics right (by, I dunno, measuring them). Just promise me this, if you pick just one thing on this list:

Select.  Good.  Leaders.

Research update

A few new journal issues have come out lately:

Summer 2016 Personnel Psychology, including:

Transparency of Assessment Centers: Low Criterion-related Validity but Greater Opportunity to Perform?

May 2016 Journal of Applied Psychology, including:

Initial impressions: What they are, what they are not, and how they influence structured interview outcomes.

Racioethnicity, community makeup, and potential employees’ reactions to organizational diversity management approaches.

June 2016 International Journal of Selection and Assessment, including:

Applicant Reactions to Selection Events: Four studies into the role of attributional style and fairness perceptions

Behavioral Cues as Indicators of Deception in Structured Employment Interviews

The Role of Self-focused Attention and Negative Self-thought in Interview Anxiety: A test of two interventions

The Influence of Candidate Social Effectiveness on Assessment Center Performance Ratings: A field study

Discrimination due to Ethnicity and Gender: How susceptible are video-based job interviews?

A Comparison of General and Work-specific Personality Measures as Predictors of Organizational Citizenship Behavior

The Perceived Nature and Incidence of Dysfunctional Assessment Center Features and Processes

Who is Being Judged Promotable: Good actors, high performers, highly committed or birds of a feather?

New journal issues

Two new journal issues to make you aware of:

International Journal of Selection and Assessment - March 2016

Unintended Consequences of Transparency During Personnel Selection: Benefitting some  candidates, but harming others?

Ethnic Differences in Perceptions of Cognitive Ability Tests: The explanatory role of self-serving attributions

Conditional Reasoning Test for Aggression: Further evidence about incremental validity

For Love or for Money: Intrinsic and extrinsic value congruence in recruitment

Social Influences in Recruitment: When is word-of-mouth most effective?

Highlighting Tensions in Recruitment and Selection Research and Practice

Tests of Integrity, HEXACO Personality, and General Mental Ability, as Predictors of Integrity Ratings in the Royal Dutch Military Police

Training Affects Variability in Training Performance Both Within and Across Jobs

Examining Applicant Reactions to Different Media Types in Character-based Simulations for Employee Selection

When Will Interviewers Be Willing to Use High-structured Job Interviews? The role of personality


Journal of Applied Psychology - March 2016


How and why do interviewers try to make impressions on applicants? A qualitative study.

The long road to employment: Incivility experienced by job seekers.

The role of self-determined motivation in job search: A dynamic approach.

How a mobile game made me re-think the nature of jobs

I have a lot of games on my iPad.  They go on for several pages as you swipe left.

Some are quite good, but most are just good enough that I can't bring myself to delete them, but not good enough that I play them regularly.

Every once in a while though, one comes along where I find myself playing it whenever I get downtime, usually due to some clever reward scheme that borrows heavily from the concept of variable interval reinforcement.  (come to think of it, most of them do)  You Candy Crush players out there know what I'm talking about.

Quite unexpectedly, I came across one recently that's not only fun, but made me re-think my approach to jobs.  Frankly it was rather eye-opening.  I know this sounds odd, but stay with me.



It's a game called Pixel People. It's a "city building" game, drawn from the rich history of games like SimCity and Civilization.  There are many games, like Clash of Clans, that have an element of city building, but unlike most of them, Pixel People doesn't focus on fighting, but instead on building your city.

The graphics aren't anything to write home about, you can see an example below.  The word "pixel" isn't in the title by accident.



Fortunately, the game doesn't need high-resolution graphics to pull off addicting gameplay.  And part of what makes it addictive--actually, the biggest part--is how you create people to populate your city.

Every citizen starts off as a clone, tabula raza.  You individualize them by giving them professions.  And the way you give them professions is by combining other professions from different categories.

For example, you start off with a Mayor and a Mechanic.  The Mayor belongs to the "Administration" occupational category, the Mechanic to the "Technical" category.  It was right about this point that something tickled at the back of my mind and I was reminded of a typical job classification system (O*NET being a prime example).

The best part of the game is that in order to move forward and create new professions and buildings, you combine two different professions.  For example, if you combine the professions of Mayor with the profession of Mechanic, you get an Engineer.  By doing so you create not only an Engineer, but access to the Garage and Mine structures.  You continue combining professions to create new ones, create new buildings which offer new abilities, etc.

Here are some other profession creation combinations, of which there are currently 400:

Mechanic + Engineer = Mechanical Engineer (duh)
Director + Model = Actor
Doctor + Park Ranger = Vet
Farmer + Farmer = Botanist
Mechanic + Police Officer = Firefighter
Architect + Dreamer = Artist

As you can see, the game designers put some thought into how different professions relate to one another.  They're not perfect, but close enough to make you smile when you create a new profession.

So what does this have to do with assessment?  Thanks for hanging in there.  Well, the game mechanic got me thinking about how overly logical and rational most of our classification systems are, and how little we acknowledge the overlap and relationships between occupations.

Most of us in HR structure our worlds around the idea that jobs can be categorized and differentiated.  And in some cases, this makes perfect sense.  A doctor is not the same as a computer programmer.  Different educational requirements.  Many core competencies required for the position are different.

But I submit to you that in many cases, occupations have more overlap than we pretend.  They are related to one another in ways that we don't typically acknowledge.  And this has implications for recruiting, assessment, compensation, promotional paths--i.e., the core work involved in talent management.  For example:

Recruiting:  currently, the ideal model of recruiting is to identify through job analysis the core KSAs or competencies required, and craft your recruitment campaign to attract those that possess them.  KSAs can get very specific, resulting in recruiting efforts often focused on a pretty narrow desired profile.  If we acknowledge that many jobs in our organizations have more overlap than we normally pretend, it becomes obvious that recruitment campaigns can become broader in two major ways: (1) you start to focus more on recruiting for the organization, not specific jobs, and (2) you start recruiting for broader skillsets or competencies, like analytical skill and conscientiousness.  I don't think it's a coincidence that the last 40+ years of assessment research has repeatedly underlined the predictive power of these qualities.

Assessment:  like recruiting, assessment strategies typically target very specific KSAs--knowledge of a particular programming language, knowledge of a particular area of HR law, etc.  If we acknowledge the overlap and relationship between jobs, it changes our assessment strategy.  Like our recruitment strategy, we focus on broader targets such as communication ability and ability to work as part of a team.  I'm not suggesting we shouldn't focus on those things that are critical to job performance and necessary upon entry to the position, but rather that we not prioritize those above more general qualities of the individual.

Compensation:  most often, particularly in civil service systems, compensation is based on the job category someone belongs to and their tenure.  If we instead acknowledge the somewhat artificial nature of our classification structures, it shifts the focus to compensation being based on contribution to the organization.  I recognize pay-for-performance has had inconsistent success, but I suspect that has as much to do with what's being compensated as it does with the concept.

Thinking about recruitment, assessment, and compensation in this way broadens our horizons when it comes to other aspects of talent management, such as career mobility.  It becomes easier to see how transferable skills benefit the organization, increasing its ability to adjust to new conditions, including unexpected turnover.  Instead of staffing focused on narrow KSAs, we fill our organization with people whose strengths allow them to move relatively fluidly between jobs, which helps the individuals as well in their career development.

Am I suggesting that we ignore specific skillsets when recruiting?  Definitely not.  Obviously sometimes you need people with a very particular ability or knowledge.  What I am suggesting is we shift the balance toward a much more inclusive perspective when it comes to the qualities we seek.

What do you think?  Has your organization already acknowledged the overlap between jobs?  Do you already recruit and select based on a broader mindset than simply those KSAs required for a particular position?  Have the long hours spent in front of a tablet warped my perspective?



Footnote: long-time readers will have noticed that I'm not posting nearly as much as I used to, and for that I apologize.  I took a new job last summer and since then my blogging has suffered.  If you want to follow me, I recommend my Facebook page, which I update more often.  Thanks for hanging in there! This year marks the 10-year anniversary of HR Tests and I hope to do something special in celebration.