January 2012 research update

Okay, we've got a lot of ground to cover so let's jump right in...

- For you stats fans: Estimating ordinal reliability for Likert-type and ordinal response data

- And more: Effect size: Don't just report it, describe it

- And still more: A confidence interval approach to testing comparability of groups

- Shifting gears: CWBs: There's more than one type

- Femininity and military leadership

- Increasing HR's strategic participation

- HR practices do have an influence on organizational attractiveness

- IQ scores may be influenced by test motivation (in low-stakes settings)

- Learning agility is an important factor in identifying high-potentials

- Slicing and dicing the Big Five

- Extraversion, intelligence, and genetics

- Apparently being agreeable won't make you rich (especially if you're a man)

- Need scientists or engineers? Consider innovativeness and internal LOC

- What if you're looking for leaders of scientists or engineers? Consider three important criteria.

- Increase diversity cues to increase recruitment website stickiness

- Does transformational leadership relate to proactive behavior? Depends on the situation.

- Core self evaluations: More complicated than you might think

- No, seriously, it's much more complicated and involves general versus specific measures

- Speaking of complicated, the relationship between personality traits (Big 5) and performance isn't as simple as many make it out to be

That's all for now! See you in February.

Final research update of 2011

Welcome to the last HR Tests research update of 2011! This year has been filled with research developments, building on previous thinking as well as venturing out into new areas. Let's see what the end of the year brings us (hint: a lot)...

We'll start with the Winter 2011 issue of Personnel Psychology:

- If you're looking for executives who excel at strategic thinking, you'll want to pay attention to not only their cognitive ability but their accumulated work history, according to Dragoni, et al.

- I tend to think of realistic job previews (RJPs) as occurring pre-hire, but research by Earnest, et al. suggests that an effective technique is to conduct an oral and written RJP post-hire--and be as honest as possible.

- Meta-analyses are relied upon heavily as summaries of large bodies of research. Roth, et al. point out ways we can make them better, particularly with respect to estimates of validity and group differences.

- One of the downsides of cognitive ability tests is they're not always perceived very well by applicants. Sumanth & Cable show how this perception is influenced by the status of the organization as well as individual status.

- Next, a fascinating description of a scale to measure the idea of calling by Dobrow and Tosti-Kharas. What I found most interesting was how the measure was associated with different criteria for different domains (e.g., art, music, management). You can read a draft version here.


Let's turn now to the January issue of JOB:

- Political skill is a hot topic, and Gentry, et al. demonstrate how the perception of promotability related to this skill varies between bosses, peers, and direct reports. Long story short: it differs, and also depends on behavior (attitude will only take you so far, right?). You can read the full version (at least right now!) here.

- Anyone that's worked with (or has in their family) engineers or scientists knows that they often share some strong traits. This suggests leading these groups that engage in creative activity may require specific attributes, which is what Robledo, et al. set out to describe.


On to the January issue of JOM:

- Core self-evaluation (CSE) is another hot topic, and Chang, et al. provide an important review of 15 years of research, including the meta-analytic support for CSE predicting in-role and extra-role performance.

- Boswell, et al. provide an integrative review of the concept of job search across different situations (e.g., following job loss, while employed).

- One area that deserves more attention is selection into the highest positions within organizations. Withers, et al. provide a review of the process of selecting a director for a board.


Let's not forget v41(2) of Personnel Review:

- Hoglund delves into the topic of talent management as a strategic HRM practice. A fascinating topic that reinforces the importance of HRM as an influence over employee perceptions and behavior.

- The wording of job ads can have important impacts on applicant perceptions and behavior. De Cooman and Pepermans analyze the differences between for-profit and non-profit job ads, and show how only a fraction of the information potentially relevant for job-person match is published.

- Another topic that deserves additional attention is the motivation to expatriate. Altman and Baruch describe results of a qualitative study that may be useful to organizations thinking about attracting and selecting for positions that require this substantial move.


Don't forget this one:

- Johnson, et al. with an important reminder that when looking at the issue of discrimination, using single categories to define groups is probably not the best strategy.


Whew! And last but certainly not least, in the December issue of IOP, Michael McDaniel and colleagues present an argument (similar to ones made elsewhere) that the Uniform Guidelines are outdated and, worse than that, a detriment to the field of selection. The commentaries are many and range from support to passionate disagreement, with a healthy dose of caution (and dare I say...intransigence?) thrown in. Worth a read, particularly for those following the Guidelines literally and those engaged in related litigation. You can read a draft version here.


I hope everyone has a great New Years; here's to a wonderful 2012!

Forget leadership assessment centers: Use the fat face test

All that time your organization spends on leadership assessment? Throw it out. New research indicates that CEOs with fat faces perform better, so just get out your measuring tape.

Okay, I'm oversimplifying. But this research by Wong and her colleagues is fascinating.

They looked at 55 photos of male CEOs and linked facial width to firm performance. Results? CEOs with higher "facial ratios" (face width relative to face height) led organizations that had "significantly greater firm financial performance."

Why facial width? Previous research has indicated this characteristic is related to aggression and sense of power. People who feel powerful tend to focus more on the big picture rather than little details, and tend to be better at staying on task.

Why men? The relationship between face ratio and behavior has been found to be important only in men. Something to do with testosterone.

But wait, don't throw out your work sample tests quite yet. The researchers also found that the relationship between facial ratio and firm performance was moderated by the style of the leadership team. The relationship was stronger in teams that were "cognitively simple" and saw things in black-or-white terms, apparently due to deference to authority. Not exactly something most organizations dream of. Presumably leadership teams with more nuanced views of the world rely on things beyond what shape their leader's head is.

Yes, I'm being tongue-in-cheek here. But if nothing else, this type of research reinforces the complex and important relationships between genetics, perceptions, and behavior.

Bad hires in hindsight

Lately I've been thinking about hires I've seen that went bad. What seemed like the right selection at the time turned into a huge disappointment, or worse--a total nightmare. This is particularly the case when selecting for supervisors and managers.

Looking back at those decisions, what happened? So far I've identified several classic cases:

1) Smallville Syndrome. While the individual may have had success in a smaller role, they were completely unfit to take on the bigger scope of the new job. The strategies that worked before are no longer adequate in the new position. This is similar to thinking your strongest specialist will make the best supervisor, but it can happen even with people that are currently in a similar role.

2) The Loch Ness Monster Effect. On the surface, everything seems fine with the applicant. But lurking below is a monster, just waiting to rear its head. Once in the right position, you will see a side of them you weren't anticipating--and it won't be good.

3) Gumby Complex. A supreme lack of flexibility leads the individual to spurn suggestions to do things differently or be open to new ideas. This often results in a toxic combination of alienating their bosses as well as their subordinates.

4) Prima donna-itis. A hard-charging and competitive individual contributor who consistently performs at the top of their group can yield big returns. But this typically isn't what you're looking for in a supervisor or manager. Instead what can happen is they build their own island where subordinates either conform or get kicked out.

5) Iwannajob condition. The individual wants a job--any job. And they don't care what they do as long as it pays. Except they do. And sooner or later they will realize this, start disengaging, and you've lost valuable training, relationships, and resources.

Turning these around, we see some ideas for improving the selection process:

1) Dig, dig, dig. Resist the urge to hire quickly. Do your homework and gather as much information as you can about potential hires--particularly ones going into leadership positions. Have the courage to not hire anyone if you don't like your candidate pool.

2) Pay attention to the minor details. Something may seem like a minor personality quirk that is overshadowed by the person's strengths. But that quirk may end up making their strengths irrelevant if their new environment causes the quirk to grow into a full-blown condition. Think about how a personality trait might exhibit itself in a new environment.

3) Focus on the job at hand. It's trendy to focus on someone's performance history. And many times this can predict performance in a new job. But when the new position requires new competencies--or a whole new level of them--what happened in the past may not be so instructive. Use a variety of strong assessment devices tied to the position, not the applicants.

4) Consider personality inventories. Yes, they can be challenging to adopt, administer, and interpret. But a lot of organizations use them successfully to help get underneath the shiny exterior. Just be very careful when selecting the particular tool to use.

5) When hiring for a supervisor or manager position, spend 80% of your time looking at personality and communication style, 20% on technical competencies. People want to be led, motivated, and engaged, not micromanaged by someone who feels like they can do the job better than you can.

6) Try to find out why the person wants the job. This isn't easy, and generally won't be had by simply asking, "So...why do you want this job?" You can start with that question, but follow it up with a bunch more that get at whether the person has really thought about their fit and what they hope to accomplish in the position.

This is of course just a sample of what can go wrong and some quick suggestions. As those of you that have been doing this a while know, hiring the right person is usually not quick and easy--particularly for leadership positions. But boy is the effort spent up front worth it. Just ask anyone who's ever had a bad boss.

Research update #583: Impression management and a lot more

Okay, I've got a lot of ground to cover this time, so buckle up...

Let's start with the December issue of IJSA:

- Looks like how much applicants try to make themselves look good varies by country

- Is applicant faking behavior related to job performance? Kinda depends on your definitions.

- Research has found that emotional intelligence can be related to work attitudes. This appears to be in part because of an increased situational judgment effectiveness.

- Speaking of situational judgment...in terms of job knowledge, knowing what to do is different than knowing what not to do

- What impact does a resume have on a recruiter? Depends on what assumptions they make about you after reading it.

- How to people select--and continue with--an executive coach? By looking at things like their ability to forge a partnership.

- How do Canadian firms do in terms of using tests other than interviews? Not so well, it turns out.


Let's move to the October issue of JASP, where there's just one article but it's a good one. Researchers continued the (depressing) finding that applicant names impact pre-interview impressions. Specifically, the more a name was Anglicized, the more favorable the impression was when hiring for an outside sales job.


Next comes the November issue of JAP:

- A new meta-analysis of the FFM of personality and its relationship to OCBs and task performance.

- Measures of interest haven't gotten a lot of love as selection devices. Looks like we need to tease out the constructs a little because they could be more helpful than we thought.

- Applicants trying to create a certain image during an interview are better off doing this after an initial flub or relying solely on self-promotion rather than making up an image.


A few from the November issue of JPSP:

- Another on impression management (not selection-specific) that goes into more detail about the topic (e.g., how many tactics people use, their accuracy)

- A caution about using the Revised NEO-PI in different cultures due to DIF.


Next, a call for more transparency in false-positive findings.


Last but not least, those of you interested in the potential of social ratings of performance being used for selection might be interested in this study of RateMyProfessors.com, which found student ratings are likely to be useful measures of teacher quality.

How important is assessment, really?

Prepare for a little blasphemy.

Over the last few months my job--and focus--has changed dramatically. Historically I've been a "testing" guy. Question about job analysis? Item writing? I'm there.

Then, a few years ago, I started managing a team that did a more than assessment--a lot more. In fact, even though assessment is in their job description, the team spends most of their time counseling supervisors on performance management. Of course some of this is because the testing workload is down, but it's also a function of demand for advice in this area.

In July we found out our department's budget was being cut; to the tune of about 10%. We adjusted and tightened our belts, but in the end it wasn't enough and we had to plan for layoffs. I was recruited to be one of the coordinators of said layoff, and thus began the dramatic work shift.

That's all a really long way of saying that my focus lately has not been on recruitment and hiring. I've been thinking a lot more about what keeps people going in difficult times. Sure, the KSAOs they bring to the table are important, but other things raise in importance during times of uncertainty and lack of control.

Which got me to thinking: how important IS assessment really? Even at our best, we can predict only about a third of individual job performance. What's going on with that other 2/3rds?

You're probably familiar with models of job performance, so I won't bore you. Suffice to say that a lot goes into job performance. So that person you hired that aced your assessments? Not guaranteed to be super star. If they end up being supervised by an incompetent manager, their inner greatness may never reach the surface. If they have a death in the family, you better believe their focus is not going to be on work for a while and job performance may not be at maximum.

Let's think about job performance as a pie. Top-notch assessment can predict about a third of that pie. What else is in that pie--and more importantly, how big are the slices? Things like:

- motivation
- supervision style
- role clarity
- co-worker support
- mood
- resources
- performance feedback
- stress

(I tried with no luck to track down a comprehensive path model, maybe one of you can point one out)

Now we can't control all of these things (although as HR professionals we certainly can consult on a lot of them--clear duty statements, supervisor training and accountability, engagement surveys, etc.), but what we CAN do is take the rigor we bring to the study of assessment and apply it to other aspects related to job performance. If you look at HR research outside of assessment I think you'll find that the level of analysis is, shall we say, sometimes lacking.

Don't get me wrong: assessment will always be important. The legal rationale is IMHO the least compelling. Instead, there is proven, substantial, utility in implementing best practices for employee assessment.

But lately I can't help but thinking: Are we spending too much time thinking about--and studying--how we bring people in to an organization, and not enough time thinking about what happens once they get there?

Advice to bballstud_23@mail.com: Apply using a different email address

It's not that I don't appreciate your desire to be different, but I worry about your chances of getting a job.

Why? Because if you apply for a job using that email address, you're going to look:

a) anti-social
b) unprofessional
c) careless
d) inexperienced

This may seem obvious, but according to a little gem tucked away in the latest TIP by Sachau and colleagues, not only are people with, shall we say, "casual" email addressed perceived negatively, they actually score lower on preemployment tests.

The authors used data from over 15,000 actual job applicants who had completed various tests from SHL. Email addresses were coded on a variety of scales such as craziness/insanity, drugs/alcohol, and sci-fi/geeky/nerdy. Then they analyzed the differences in test scores between people with more professional emails with the more casual ones.

Not only did raters agree on the lack of professionalism these types of addresses exhibited, individuals with these addresses actually performed lower on measures of conscientiousness, professionalism, work experience, and total score.

And while the effect sizes weren't huge (there was about a 10% difference in group means between high and low groups), it should give you pause that even if your address is questionable as opposed to outright silly, you will be perceived as unprofessional and may be assumed to have lower test scores.

...unless that's what you're going for.