Sunday, May 22, 2016

First, get the people basics right

Competencies, talent, gamification... there's no doubt about it, we like us some buzzwords.  Like bright shiny objects, these ideas entice--and largely detract.

Sometimes new ideas and ways of thinking can lead to significant improvements in the way organizations manage their people. But here's the truth that no one seems to want to talk about: many organizations fail to get the basics right.  So while leaders may be leaping headlong into the nanofied virtual talent management sunset, the foundation of HR is lacking.

What are these basics of which I speak?

1. Adequately defining jobs--based on subject matter expert data.   Every single job should be defined and documented in terms of key tasks, requirements, and expectations.  The form this takes is less important than the quality of the data. This is the bedrock that helps you recruit, select, reward, and manage effectively.

2. Recruiting like you're selling, not like you're being forced to.   Writing attractive job ads is so easy, why aren't we swimming in them?  The same reason many organizations fail to accurately describe the job: laziness and lack of discipline.

3. Using valid hiring measures.  Speed of hire is important, but not even remotely as important as quality.   I can make you a sandwich really quickly if it's just bread.   Do you think Google gets millions of resumes each year because candidates are hoping for a quick hire?  Importantly, the higher in the organization, the more time should be spent on valid assessment.

4. Holding leaders accountable for being leaders. This really should be #1 except I was trying to go chronologically (and will fail miserably).  All too often, it's the line staff who are quickly called to the carpet when they make mistakes.   But holding leaders accountable for their behavior (hint: ask their subordinates) is exponentially more powerful.

5.  Listening to each other.  Many if not most good ideas for improving your organization are in the heads of your line staff.   Do you ask them regularly and implement their ideas?  Is listening skill considered critical for all employees?

6.  Saying thank you.  It's easy, it's cheap.   Do it more, and mean it.  

7.  Dealing firmly with poor performance.  This is top to bottom, from not being helpful on the phone to running productive meetings.  Again, the higher in the organization, the more important this is.

8.  Growing your people--forever.  Sure,  they may leave, but they'll leave sooner if you don't invest in them.  And like everything else on this list, it grows your reputation. 

9. Treating people with respect and fundamental human decency.   If you have this as a backbone, many other things simply follow.  There's a reason why one of the most popular business books recently is The No Asshole Rule.


None of this is incredibly difficult, it just takes the most precious resource of any organization: time.   And it takes commitment and discipline.  But these aren't initiatives.   They're part of an organization's DNA--or not.  They're how people respond when asked what it's like to work there.   And who is responsible for ensuring they happen ?  The people at the top. 

So before your organization jumps on to the latest buzzword bandwagon, make sure it's getting these basics right (by, I dunno, measuring them). Just promise me this, if you pick just one thing on this list:

Select.  Good.  Leaders.

Wednesday, May 04, 2016

Research update

A few new journal issues have come out lately:

Summer 2016 Personnel Psychology, including:

Transparency of Assessment Centers: Low Criterion-related Validity but Greater Opportunity to Perform?

May 2016 Journal of Applied Psychology, including:

Initial impressions: What they are, what they are not, and how they influence structured interview outcomes.

Racioethnicity, community makeup, and potential employees’ reactions to organizational diversity management approaches.

June 2016 International Journal of Selection and Assessment, including:

Applicant Reactions to Selection Events: Four studies into the role of attributional style and fairness perceptions

Behavioral Cues as Indicators of Deception in Structured Employment Interviews

The Role of Self-focused Attention and Negative Self-thought in Interview Anxiety: A test of two interventions

The Influence of Candidate Social Effectiveness on Assessment Center Performance Ratings: A field study

Discrimination due to Ethnicity and Gender: How susceptible are video-based job interviews?

A Comparison of General and Work-specific Personality Measures as Predictors of Organizational Citizenship Behavior

The Perceived Nature and Incidence of Dysfunctional Assessment Center Features and Processes

Who is Being Judged Promotable: Good actors, high performers, highly committed or birds of a feather?

Sunday, February 28, 2016

New journal issues

Two new journal issues to make you aware of:

International Journal of Selection and Assessment - March 2016

Unintended Consequences of Transparency During Personnel Selection: Benefitting some  candidates, but harming others?

Ethnic Differences in Perceptions of Cognitive Ability Tests: The explanatory role of self-serving attributions

Conditional Reasoning Test for Aggression: Further evidence about incremental validity


For Love or for Money: Intrinsic and extrinsic value congruence in recruitment

Social Influences in Recruitment: When is word-of-mouth most effective?

Highlighting Tensions in Recruitment and Selection Research and Practice

Tests of Integrity, HEXACO Personality, and General Mental Ability, as Predictors of Integrity Ratings in the Royal Dutch Military Police

Training Affects Variability in Training Performance Both Within and Across Jobs

Examining Applicant Reactions to Different Media Types in Character-based Simulations for Employee Selection

When Will Interviewers Be Willing to Use High-structured Job Interviews? The role of personality


Journal of Applied Psychology - March 2016


How and why do interviewers try to make impressions on applicants? A qualitative study.

The long road to employment: Incivility experienced by job seekers.

The role of self-determined motivation in job search: A dynamic approach.


Sunday, January 24, 2016

How a mobile game made me re-think the nature of jobs

I have a lot of games on my iPad.  They go on for several pages as you swipe left.

Some are quite good, but most are just good enough that I can't bring myself to delete them, but not good enough that I play them regularly.

Every once in a while though, one comes along where I find myself playing it whenever I get downtime, usually due to some clever reward scheme that borrows heavily from the concept of variable interval reinforcement.  (come to think of it, most of them do)  You Candy Crush players out there know what I'm talking about.

Quite unexpectedly, I came across one recently that's not only fun, but made me re-think my approach to jobs.  Frankly it was rather eye-opening.  I know this sounds odd, but stay with me.



It's a game called Pixel People. It's a "city building" game, drawn from the rich history of games like SimCity and Civilization.  There are many games, like Clash of Clans, that have an element of city building, but unlike most of them, Pixel People doesn't focus on fighting, but instead on building your city.

The graphics aren't anything to write home about, you can see an example below.  The word "pixel" isn't in the title by accident.



Fortunately, the game doesn't need high-resolution graphics to pull off addicting gameplay.  And part of what makes it addictive--actually, the biggest part--is how you create people to populate your city.

Every citizen starts off as a clone, tabula raza.  You individualize them by giving them professions.  And the way you give them professions is by combining other professions from different categories.

For example, you start off with a Mayor and a Mechanic.  The Mayor belongs to the "Administration" occupational category, the Mechanic to the "Technical" category.  It was right about this point that something tickled at the back of my mind and I was reminded of a typical job classification system (O*NET being a prime example).

The best part of the game is that in order to move forward and create new professions and buildings, you combine two different professions.  For example, if you combine the professions of Mayor with the profession of Mechanic, you get an Engineer.  By doing so you create not only an Engineer, but access to the Garage and Mine structures.  You continue combining professions to create new ones, create new buildings which offer new abilities, etc.

Here are some other profession creation combinations, of which there are currently 400:

Mechanic + Engineer = Mechanical Engineer (duh)
Director + Model = Actor
Doctor + Park Ranger = Vet
Farmer + Farmer = Botanist
Mechanic + Police Officer = Firefighter
Architect + Dreamer = Artist

As you can see, the game designers put some thought into how different professions relate to one another.  They're not perfect, but close enough to make you smile when you create a new profession.

So what does this have to do with assessment?  Thanks for hanging in there.  Well, the game mechanic got me thinking about how overly logical and rational most of our classification systems are, and how little we acknowledge the overlap and relationships between occupations.

Most of us in HR structure our worlds around the idea that jobs can be categorized and differentiated.  And in some cases, this makes perfect sense.  A doctor is not the same as a computer programmer.  Different educational requirements.  Many core competencies required for the position are different.

But I submit to you that in many cases, occupations have more overlap than we pretend.  They are related to one another in ways that we don't typically acknowledge.  And this has implications for recruiting, assessment, compensation, promotional paths--i.e., the core work involved in talent management.  For example:

Recruiting:  currently, the ideal model of recruiting is to identify through job analysis the core KSAs or competencies required, and craft your recruitment campaign to attract those that possess them.  KSAs can get very specific, resulting in recruiting efforts often focused on a pretty narrow desired profile.  If we acknowledge that many jobs in our organizations have more overlap than we normally pretend, it becomes obvious that recruitment campaigns can become broader in two major ways: (1) you start to focus more on recruiting for the organization, not specific jobs, and (2) you start recruiting for broader skillsets or competencies, like analytical skill and conscientiousness.  I don't think it's a coincidence that the last 40+ years of assessment research has repeatedly underlined the predictive power of these qualities.

Assessment:  like recruiting, assessment strategies typically target very specific KSAs--knowledge of a particular programming language, knowledge of a particular area of HR law, etc.  If we acknowledge the overlap and relationship between jobs, it changes our assessment strategy.  Like our recruitment strategy, we focus on broader targets such as communication ability and ability to work as part of a team.  I'm not suggesting we shouldn't focus on those things that are critical to job performance and necessary upon entry to the position, but rather that we not prioritize those above more general qualities of the individual.

Compensation:  most often, particularly in civil service systems, compensation is based on the job category someone belongs to and their tenure.  If we instead acknowledge the somewhat artificial nature of our classification structures, it shifts the focus to compensation being based on contribution to the organization.  I recognize pay-for-performance has had inconsistent success, but I suspect that has as much to do with what's being compensated as it does with the concept.

Thinking about recruitment, assessment, and compensation in this way broadens our horizons when it comes to other aspects of talent management, such as career mobility.  It becomes easier to see how transferable skills benefit the organization, increasing its ability to adjust to new conditions, including unexpected turnover.  Instead of staffing focused on narrow KSAs, we fill our organization with people whose strengths allow them to move relatively fluidly between jobs, which helps the individuals as well in their career development.

Am I suggesting that we ignore specific skillsets when recruiting?  Definitely not.  Obviously sometimes you need people with a very particular ability or knowledge.  What I am suggesting is we shift the balance toward a much more inclusive perspective when it comes to the qualities we seek.

What do you think?  Has your organization already acknowledged the overlap between jobs?  Do you already recruit and select based on a broader mindset than simply those KSAs required for a particular position?  Have the long hours spent in front of a tablet warped my perspective?



Footnote: long-time readers will have noticed that I'm not posting nearly as much as I used to, and for that I apologize.  I took a new job last summer and since then my blogging has suffered.  If you want to follow me, I recommend my Facebook page, which I update more often.  Thanks for hanging in there! This year marks the 10-year anniversary of HR Tests and I hope to do something special in celebration.

Thursday, August 13, 2015

Where there's a will, there's a way: OPM shows how to do UIT the right way


On August 10 and 12, PTC-NC was privileged to have Dr. Patrick Sharpe from the U.S. Office of Personnel Management (OPM) deliver a presentation about USA Hire, part of a suite of online platforms that has allowed OPM to revolutionize the way they deliver assessment services to their diverse customers.

The idea to ask Dr. Sharpe to present occurred to me when I read an article in the Washington Post on April 2 about USA Hire titled, "For federal-worker hopefuls, the civil service exam is making a comeback".   It provides an overview of what OPM has managed to accomplish with its partner, PDRI, in the area of unproctored internet testing (UIT).  Start there if you want to learn more because it includes some examples of the items--although significantly absent is an example of the excellent video avatar-based assessment used for things like situational judgment.

Dr. Sharpe did an excellent job painting the picture for the audience of how much work was involved in the project, and how important things like stakeholder communication and contract management were to ensuring the project was a success.  He then showed us a demo version of USA Hire, where he lead us through what it looks like from the applicant's perspective as they proceed through a series of competency-based assessments.  The item formats range from the traditional (e.g., reading comprehension multiple-choice) to the modern (avatar-based SJT) to the groundbreaking--at least for the public sector (forced-choice non-cognitive assessment).

Here are some of the key points I took away:

- The technology is just a part of successfully putting an UIT program together, you have to step back and look first at what you're trying to accomplish.  For example, are you interested in whole person assessment (as OPM is) or simply focusing on certain KSAs?

- USA Hire is the culmination of years of research and analysis, and traces its history back 20-30 years within the federal government.   Translation : don't jump into UIT without careful planning.

- Start with the basics when delivering UIT: make sure the customer has a solid job analysis foundation before jumping to the assessment platform

- Getting a larger, more influential, customer successfully implemented can cause others to jump on board

- Realize that, particularly in a decentralized testing environment, you may still end up with a hybrid of different testing approaches following the roll-out of UIT, and this includes T&Es.  But the best way to move the practice is to show what success looks like.

- Consider carefully whether you want to build, buy, or lease the technology.  There are benefits and drawbacks to each.

- Starting with a pilot can be a great way to test the system (no pun intended), and also demonstrate the potential to stakeholders.

- Collaboration between assessment professionals, HR specialists, and vendors is critical.

- Don't underestimate the importance of change management.  Fears (e.g., about losing control) come easily and have to be addressed head-on.

- Organizational and system readiness is very important.  Part of the reason this effort was successful is because hiring organizations were fed up with the extremely low utility (and perception) of point-based T&Es.

For someone passionate about assessment and technology, the presentation was educational and motivational.  I walked away, as did others, with a new-found optimism for what sufficient will, resources, and tenacity can accomplish.  It's seductive to focus on what can't be done in the public sector, so to hear and see what can be done reemphasizes the importance of leadership--both in HR and at the top of the organization.

Unproctored internet testing has been talked about for so long, but to see it in action, in a research-based way in the public sector, is truly inspirational.  Truly where there is a will, there is a way.

Saturday, March 14, 2015

The secret to successful organizations: Let HR drive hiring



Why do organizations make bad hires?  Because they rely too much on supervisors to do the hiring.

Imagine this scenario.  You take your car into the shop because it's making a strange sound.  The mechanic fixes it, you pay, and you're about to leave.  But on your way out, the mechanic stops you.  She* notices that you're frustrated with your iPad because it's being sluggish.  She offers to fix it for you, reasoning that cars and iPads can't be all that different.

Would you take the mechanic up on her offer?

Of course not.  But handing over important decisions--arguably the most important decisions--to the wrong people is something organizations do every day.

Auto mechanics are trained to deal with a specific situation, and to do it successfully: fix cars.  Their education and experience prepares them to do so.

Similarly, supervisors are trained to do primarily one thing: supervise the day-to-day work.  They're usually promoted because they excelled at the line level (i.e., they understand the work) and they show aptitude for leadership (hopefully).  They are not, generally speaking, schooled or experienced with the professional side of personnel selection.

Am I suggesting supervisors NOT be involved?  Absolutely not.  Am I suggesting that all HR shops are staffed with experts in personnel assessment and measurement?  Nope.

What I am suggesting is organizations get serious about this issue and stop treating supervisors as if they are people measurement professionals.

In the modern workplace with information overload and time at a premium, it's tough enough to ensure that hiring and promotion decisions get made in a thoughtful fashion.  The problem is exacerbated by the fact that most supervisors overestimate their hiring ability.  After all, how hard can it be to interview someone?

Not hard, if you don't care about getting it right.  But getting it right requires careful thought and planning; hiring right is not something done at the last minute without regard to competencies that drive success in the specific position.  It's not something that looks exactly the same time.  It's tailored to the position, the culture, and the particular needs at the time.

Great hires happen systematically for one reason: a tight partnership between line supervisors and talented HR consultants.  The supervisor knows the job. They often know best what competencies are needed to perform the job successfully (although a good internal HR consultant will have a pretty good idea).  HR professionals are trained in the professional side of selection, recognizing the pitfalls and identifying those measures most likely to predict success.

"But our HR shop doesn't know what it's doing.  They don't have the expertise!" some might say.

Perhaps.  If so, you've just identified your second strategic problem.  And it should be fixed.  That's like having people in your budget office that are bad at math.

But consider this: you may be surprised to learn that many hiring supervisors welcome being removed as the primary driver of hiring.  Good supervisors recognize that this isn't their greatest strength, and they will be happy to have HR assist them in identifying the most qualified applicants.  Particularly in this age of online questionnaires and massive candidate lists for many jobs.

So who is responsible for ensuring this supervisor-HR partnership happens systematically and is built into the organizational culture?  The leaders at the top.  Director.  CEOs.  They're the ones minding the store, and they're ultimately responsible for organizational success.  They should be instituting policies, procedures, and cultural norms that emphasize how important and critical hiring is--so critical that it requires a team approach to get it right.

Anything else is simply not taking the success of the enterprise seriously.


* Did I throw you with a female mechanic?  Now might be a good time to take the implicit association test for gender and careers.

Thursday, March 05, 2015

Mega research update

I hope you like research, because there's a lot of it coming your way...and many are free as of this posting!

Without further ado...


Let's start with the Journal of Applied Psychology, January 2015 issue:

- We see a lot of research involving large candidate groups, but much less for individuals.  In this meta-analysis of individual assessments, the authors found support for their usefulness, but it varied significantly across studies.  Highest validity was found for managerial jobs and assessments that included a cognitive ability test.

- Being in the wrong job can be frustrating for both the employee and the employers.  In this study, the authors show a relationship between poor vocational fit and counterproductive work behaviors (CWBs).

- Speaking of CWB, there may be more of them going on than you would think based on the assessment literature...

- And even more on CWB!  These authors found support for both self- and acquaintance-reported personality ratings, specifically conscientiousness and agreeableness, in predicting "workplace deviance".

- Unfortunately, gender bias still exists in selection.  In this meta-analysis, the authors found this to be particularly the case in male-dominated jobs.  On a positive note, they do suggest ways of mitigating this: provide clear evidence of the competence of applicants, encourage careful decision making, and use experienced raters.

- The over-/under-prediction of cognitive ability tests debate for different ethnic groups continues.  In this study, the authors find support of overprediction for African Americans, suggesting the tests are not predictively biased.



Next, the March issue of J.A.P.:

- More support for the predictive validity of emotional intelligence, but more importantly, how the concept overlaps with other constructs such as the Big 5 and self-efficacy.

- All situational judgment tests (SJTs) are not equal, and according to these authors in a large number of instances the context that is presumably important?  Not so much.

- Speaking of SJTs, these researchers suggest that putting the "situational" back in SJTs--i.e., assessing how the situation is analyzed rather than the response options--is a useful method.

- A fascinating update of effect size benchmarks that can be used for a variety of purposes.

- Trying to predict safety-related behavior?  This research suggests that personality traits, particularly agreeableness, can usefully predict this behavior.


Moving on to the March issue of IJSA (free right now!):

- Some guidelines on preparing norms for personality inventories.

- Evidence that different cultures have different procedural justice perceptions of different selection mechanisms

- Some important findings on the equivalence and stability of job performance ratings over time

- Development of a new measure of subjective career success

- More evidence that both technical knowledge and prosocial knowledge are important factors in predicting medical student clinical performance

- This study found that CWBs are under-reported and organizational commitment increases the likelihood that peers will report them

- Evidence that forced-choice and Likert-type scales used in personality inventories have similar measurement properties


On to the Spring issue of Personnel Psych (also free right now!):

- This meta-analysis on narcissism showed that it is related to leadership emergence (through extraversion) and leadership effectiveness in a curvilinear fashion.

- More evidence of the importance of political skill--particularly the aspects of networking ability, interpersonal influence, and apparent sincerity--in predicting a range of important outcomes, including task performance beyond GMA and the Big 5.  It would be interesting to see how this is related to emotional intelligence (yes this is a foreshadowing).


Turning to the March issue of Psych Bulletin:

- More on narcissism: this time, researchers found that men consistently report higher levels of narcissism compared to women, which is interesting when taken in combination with the study above.


In the December issue of Industrial and Organizational Psychology:

- The first focal article calls out researchers for using incorrect assumptions about criterion reliabilities, thus impacting criterion validity values.  They make suggestions for how to improve meta-analyses moving forward.

- The second makes the important argument that utility analyses should consider measures of well-being when determining the effectiveness of interventions (such as an employment test).


Finally, in the January issue of JOB (also free right now):

- a proposal for improving the calculation and reporting of Cronbach's alpha

- a fascinating study showing that high conscientiousness may hinder performance during stressful situations

- in support of EI, this study found a link between emotion recognition ability and income (interestingly through political skill and interpersonal facilitation...remember the earlier study on political skill?).


That's all for now!