Saturday, July 30, 2016

Welcome to Sacramento, IPAC'ers!

This year, IPAC's annual conference is here in my home of Sacramento from July 31 - August 3.  I'll be there Monday afternoon for a session titled "Fits and Starts: The Evolution of Testing for the State of California (Special Invited Session)", where I'll be interviewing my good friend, Adria Jenkins-Jones.  Here's the description:

In this lively discussion, the presenters will discuss the current state of employment testing for the state of California, including significant recent and upcoming changes to our examination software. Using an interview format, the presenters will discuss the massive changes envisioned for statewide testing, and how the California Department of Human Resources is attempting to collaborate with stakeholders to change the traditional paradigm, and significantly improve automation, while maintaining the commitment to merit. The presenters will engage in open dialogue and there will be time for audience members to ask questions about current and future directions of testing for the state.

Feel free to use the comments section below to post the sessions you plan on attending, or anything about your experience!

Saturday, July 02, 2016

Hiring for goodwill (not Goodwill)


Those of us that write, teach, and consult on personnel assessment usually paint the process as a very rational process broken into several steps:

Step 1) Study the job to identify the critical tasks

Step 2) Identify the knowledge, skills, and abilities (or competencies) required day one to perform those tasks

Step 3) Create selection systems that accurately measure candidates' levels of #2

Step 4) Hire the person(s) who demonstrate the highest scores on #3

But as you know, hiring is rarely so logical.  In some cases this is due to organizations not having the expertise, or making the time, to follow the steps above.

But in other cases, there are simply other factors at play--factors that can't be ignored by the decision makers because they exert such a significant influence on the process.  And these factors are largely intangible, meaning they're more difficult to quantify and therefore not typically included explicitly in a standard assessment process.

For the purposes of discussion, I'm not talking about negative factors, such as illegal discrimination or selections based purely on politics.  Instead, let's think of potentially useful constructs.

For example:

1) Physical presence related to professionalism.  Let's be honest, looks matter when it comes to interviews.  In general, polished shoes beat flip flops.  Suits beat sweats.  No scent beats heavy cologne/perfume.  But this may not be given a formal rating during the process--for several reasons, including the difficulty of rating and the potential overlap with discriminatory considerations.  But again, that doesn't mean it's not a factor.

2) Honesty.  How many times have you been on an interview panel and perceived a candidate more positively because he/she was honest--about a weakness, for example, or about what they need to be successful in the position.  It matters, and one reason it matters is because so many candidates hide behind a veneer of perfection.  But are you giving them a formal rating on honesty?

3) Potential.  The applicant may not have demonstrated the KSAs that are key for the job, and thus may score poorly on assessments that measure past performance.  But that doesn't mean they aren't capable of learning those KSAs, and even exceeding the performance of those that have already mastered them.  Why?  Because individual performance is more than just the KSAs someone brings to the job--it's also about the organizational environment, onboarding, development opportunities, the supervisor, and other factors that often go unmeasured during the hiring process (e.g., personality).

4) Preexisting relationships.  It's quite common for organizations to seek out individuals who have existing relationships that can be capitalized on.  For example, someone may have relationships with potential customers, or may be able to access a market in a different way.  If an organization is trying to enhance its relationship with certain key stakeholders, they may seek out someone who already has established connections.  Sales professions immediately comes to mind, but other types, such as consultants or leadership positions, may also benefit greatly from these connections.

In many ways these factors can be thought of as individual goodwill, similar to the accounting principle.  In this context, an organization's brand is an example of an intangible asset that nonetheless has great value.  And in a similar way, individual goodwill is difficult to quantify.

So what is someone that cares deeply about hiring right to do about all this?

First of all, acknowledge that these factors play a role.  Any attempts to link your assessments to outcomes that don't take these factors into account is missing a huge potential explanatory factor.  You may draw conclusions regarding your assessments that are simply false because of the nature of the selection decision.

Second, attempt whenever possible to build these intangibles into the hiring process.  For example, preexisting relationships could be considered part of a communication competency.  If hiring for potential, break down what the core abilities are that you're considering for potential--chances are you can measure those.  There are both overt and covert measures of honesty and integrity--that work.

Third, as an assessment community, we should all recognize that these factors play a role.  When we research, teach, and consult on hiring, acknowledge these factors, and help others understand how to take them into consideration when designing a successful selection system.

In the end, it's a relatively simple prescription: let's make the intangible, tangible.

Saturday, June 11, 2016

One way to reduce interviewer leniency/severity


A persistent challenge in interviews is that certain interviewers tend to be lenient (i.e., score candidates highly) while others are consistently more severe (i.e., score candidates lower).  This of course is not ideal as it introduces measurement bias as well as reduces the defensibly of the process.

One way to reduce these tendencies discussed by Hartwell and Campion in the June 2016 issue of Journal of Applied Psychology is to provide interviewers with what they call "normative feedback interventions."  Basically what this means is giving interviewers data on how they rate candidates over time compared to how other interviewers rated.  It can reveal to interviewers that they tend to rate candidates more harshly, or more easily, than others.

What Hartwell and Campion found in their study (of over 20,000 interviews using more than 100 interviewers) is that by providing this feedback to interviewers, it minimized interviewer differences and increased interview reliability--both obviously good things in terms of quality of the process.  Interestingly, it did not seem to impact the validity of the interviews, but it did impact which particular candidates were hired.

Up until now, one of the most often recommended practices for reducing rating errors has been pre-interview instructions and guidance regarding these errors.  What this study suggests is we can do even better by providing interviewers with objective data about their ratings over time.  Listening to someone talking about rating bias probably feels a lot different than actually seeing how you do compared to your peers!

Wednesday, June 01, 2016

Some easy tests to improve your hiring success

The interview is such a commonly used hiring assessment that it's hardly worth mentioning (although there is always room for improvement).


But what if you're already doing interviews and you want some easy to implement add-ons?  No problem.  Here are some ways to improve quality of hire for knowledge worker positions that don't take a long time, an automated solution, or a PhD to develop:


1. Pre-screening questionnaire.  Whether you use something quick and cheap like SurveyMonkey or your own proprietary assessment system, it's easy to create open- and closed-ended items that serve to screen out the uninterested, allow you to get some more detail from candidates, and even help you solve problems you've been struggling with!  Keep it relatively short so you don't dissuade the most in-demand candidates.


2. Targeted cover letter.  Don't just ask for a generic cover letter, ask applicants to describe in their letter how their background syncs with the core competencies you're looking for.  Remember: limit the length; two pages is generally sufficient.



3. Research project.  As part of the application process, as candidates to look into an issue that's relevant for the job.  How do they think the new overtime regulations will impact the industry?  What new technologies are on the horizon that will change the way this job is done?  Have them briefly write up their results, and/or ask about it during your interview.


4. Writing exercise.  There's no substitute for live demonstrations of writing ability.  Have them correct a document you've messed up, ask them to write a quick memo to a customer--just something related to the core duties of the job that you would expect them to be able to do day one.


5. Rule/procedure application.  Knowledge worker jobs are characterized by frequent application of laws, rules, and procedures to specific situations.  Either provide them with the rules ahead of time or give them the short version, then give them a specific fact pattern and have them come up with a solution or options.



6. Oral presentation.  Have you ever been on a hiring panel where there's an oral presentation?  If not, you're missing out.  Presentations are a great way to mix things up and see those different skillsets that a candidate brings.  Of course realize that you're adding a presentation to an interview, which I'm pretty sure are both in the top five of most stressful events.


Notice that you can mix and match theses approaches: have them do a rule/procedure application and then write a memo.  Have them do an oral presentation on a topic they researched ahead of time.  Of course, the tests you use should be based on the requirements of the job; start with entry level KSAs needed and let the assessments flow from that.  Beyond that, be creative!

Sunday, May 22, 2016

First, get the people basics right

Competencies, talent, gamification... there's no doubt about it, we like us some buzzwords.  Like bright shiny objects, these ideas entice--and largely detract.

Sometimes new ideas and ways of thinking can lead to significant improvements in the way organizations manage their people. But here's the truth that no one seems to want to talk about: many organizations fail to get the basics right.  So while leaders may be leaping headlong into the nanofied virtual talent management sunset, the foundation of HR is lacking.

What are these basics of which I speak?

1. Adequately defining jobs--based on subject matter expert data.   Every single job should be defined and documented in terms of key tasks, requirements, and expectations.  The form this takes is less important than the quality of the data. This is the bedrock that helps you recruit, select, reward, and manage effectively.

2. Recruiting like you're selling, not like you're being forced to.   Writing attractive job ads is so easy, why aren't we swimming in them?  The same reason many organizations fail to accurately describe the job: laziness and lack of discipline.

3. Using valid hiring measures.  Speed of hire is important, but not even remotely as important as quality.   I can make you a sandwich really quickly if it's just bread.   Do you think Google gets millions of resumes each year because candidates are hoping for a quick hire?  Importantly, the higher in the organization, the more time should be spent on valid assessment.

4. Holding leaders accountable for being leaders. This really should be #1 except I was trying to go chronologically (and will fail miserably).  All too often, it's the line staff who are quickly called to the carpet when they make mistakes.   But holding leaders accountable for their behavior (hint: ask their subordinates) is exponentially more powerful.

5.  Listening to each other.  Many if not most good ideas for improving your organization are in the heads of your line staff.   Do you ask them regularly and implement their ideas?  Is listening skill considered critical for all employees?

6.  Saying thank you.  It's easy, it's cheap.   Do it more, and mean it.  

7.  Dealing firmly with poor performance.  This is top to bottom, from not being helpful on the phone to running productive meetings.  Again, the higher in the organization, the more important this is.

8.  Growing your people--forever.  Sure,  they may leave, but they'll leave sooner if you don't invest in them.  And like everything else on this list, it grows your reputation. 

9. Treating people with respect and fundamental human decency.   If you have this as a backbone, many other things simply follow.  There's a reason why one of the most popular business books recently is The No Asshole Rule.


None of this is incredibly difficult, it just takes the most precious resource of any organization: time.   And it takes commitment and discipline.  But these aren't initiatives.   They're part of an organization's DNA--or not.  They're how people respond when asked what it's like to work there.   And who is responsible for ensuring they happen ?  The people at the top. 

So before your organization jumps on to the latest buzzword bandwagon, make sure it's getting these basics right (by, I dunno, measuring them). Just promise me this, if you pick just one thing on this list:

Select.  Good.  Leaders.

Wednesday, May 04, 2016

Research update

A few new journal issues have come out lately:

Summer 2016 Personnel Psychology, including:

Transparency of Assessment Centers: Low Criterion-related Validity but Greater Opportunity to Perform?

May 2016 Journal of Applied Psychology, including:

Initial impressions: What they are, what they are not, and how they influence structured interview outcomes.

Racioethnicity, community makeup, and potential employees’ reactions to organizational diversity management approaches.

June 2016 International Journal of Selection and Assessment, including:

Applicant Reactions to Selection Events: Four studies into the role of attributional style and fairness perceptions

Behavioral Cues as Indicators of Deception in Structured Employment Interviews

The Role of Self-focused Attention and Negative Self-thought in Interview Anxiety: A test of two interventions

The Influence of Candidate Social Effectiveness on Assessment Center Performance Ratings: A field study

Discrimination due to Ethnicity and Gender: How susceptible are video-based job interviews?

A Comparison of General and Work-specific Personality Measures as Predictors of Organizational Citizenship Behavior

The Perceived Nature and Incidence of Dysfunctional Assessment Center Features and Processes

Who is Being Judged Promotable: Good actors, high performers, highly committed or birds of a feather?