You only have two more days to take advantage of the "early bird" registration rate for the 2012 International Personnel Assessment Council (IPAC) conference in Las Vegas on July 22-25.
This year's conference promises to uphold the high quality of presentations and opportunities to network that you've come to expect. Great keynote speakers are lined up, such as Wayne Cascio, Kevin Murphy, and David Campbell.
Check out the flyer and the preliminary program to see all the goodness!
Tuesday, June 26, 2012
Saturday, June 09, 2012
I hate resumes
Okay, I know last time I talked about how I hate interviews--or at least how they're used. I promise this blog isn't turning into 100% rants. But some things ya just gotta get out.
My dislike for interviews is far eclipsed by my distaste for resumes. In fact the process of reviewing resumes as a selection device is far worse than most interviews.
Why? Because, like interviews, resume requirements are ubiquitous, but research has shown their validity (i.e., the relationship between resume scoring future job performance) is worse.
Aside from straight up not working very well as an assessment tool, what else is wrong with resumes? Here's a sample:
- They're not standardized. You'll get some information on some resumes, different information on others. So you end up comparing apples and oranges. Not exactly what you hope for in terms of reliability.
- They don't lend themselves easily to a scoring rubric. Sure, you can try giving a certain number of points for a degree or a certain amount of experience (assuming you can support it), but that's assuming the person writes it down in the way you need...Otherwise, it's easy to let them fall into "piles" (definitely yes, maybe, hell no). Unfortunately at that point you're essentially saying there is no variance within each pile, which is obviously not true.
- They're open to all kinds of bias, ranging from racial discrimination to wild assumptions about education ("hey, look--they went to the same school I did! cool!")
- They're time consuming to review. We've all seen resumes the size of small novels (hint to job seekers: this does not impress most employers). Separating the wheat from the chaff is about as much fun as a 4:30 meeting.
- Like training and experience (T&E) measures, they tell you nothing about how well someone did something, just that they did it. Assuming they're telling the truth, which leads to...
- You're forced to take on faith that the education and experience they describe was actually obtained. Sure, you'll verify it (right?), but at the very least you've wasted time including liars/embellishers in the process.
- The information you need may not be there. You will almost certainly find yourself thinking, "gosh, I wish they had given me more detail about X..."
- On the other hand, they contain a lot of information you don't need. Applicants try to put their best foot forward and are also guessing what you're looking for, so they waste their time and yours. They're just plain inefficient--for applicants and employers.
So why do employers use them? For a lot of the same reasons they recycle interviews:
- It's easy; heck, just put "resume required" on the ad...because...well...why not? It doesn't require us to think very deeply about the recruitment.
- It's the devil we know. It's what people are comfortable with. People may acknowledge their imperfection, but this personality testing stuff sounds scary...
- Inertia. Yes, unfortunately "it's what we've always done" has an inordinate amount of influence on the way people in organizations make decisions.
- A lack of attention to and focus on the process. The reason many selection processes are lackluster is because the hiring supervisors don't take it seriously enough. Who's fault is this? Well theirs, obviously, but HR's as well, and upper management. Hiring should be one of the most important things employees do.
- They don't know any better. This is perhaps the saddest reason because it's simply due to a lack of education about selection. Many hiring supervisors are unaware of the range of assessments out there, or maybe they've heard vague rumors about how they can be sued if they use an intelligence test. HR and upper management, I'm looking at you again on this one.
- They're not held accountable for the quality of their hires and the selection process they're using. Heck if requiring resumes is easy, it seems to mostly work, and no one's the wiser, why would I change my ways?
So aside from educating supervisors and holding them accountable, what can be done? There have been several web-based attempts to make resumes more relevant. But so far there is no "killer app" that both job hunters and employers make a beeline for. There's no Facebook of resumes (and no, Facebook is not the Facebook of resumes). In the meantime, here are some tips:
- Don't default to asking for a resume. Really think about what you're looking for. Interested in their job-related education, training, and experience? Then ask for that! Like...
..."Specifically describe how your experience and training matches the requirements of the position..."
...Or even better: "This job requires an advanced ability to put peanut butter on bread. Describe specifically how your training and experience qualifies you to perform this task."...
- Don't assume that standard applications solve the problem. They tend to have a lot of the same problems--e.g., the candidate gets total flexibility in describing their previous jobs.
- If you need, for whatever reason, to ask for a resume, try asking for it in a specific (chronological, functional) format. And be clear about what it should--and should not--contain. And limit the length. Who knows, it may at least speed things up for you.
Resumes and interviews. Probably the most frequently used personnel selection tools used. And sadly in many cases severely lacking in validity. Really the problem gets us back to the core of good selection: taking hiring seriously and spending time thinking about job requirements and a plan for assessing them.
These steps aren't that hard, and simply taking a more critical look at them may give you more payoff than any other revamp of your hiring and promotion process.
Tuesday, June 05, 2012
I hate interviews
I hate interviews.
Let me clarify. I don't hate doing interviews, I hate how they're used.
Try this exercise. Spread your arms out as far as they'll go (or envision it). That's how much of a person's competencies employers think they are measuring using the interview.
Now put your hands about a foot apart from each other. That's how much of a person's competencies they're really measuring. And that's if they interview well (which many don't).
So why do employers like interviews so much? Yes, when done right they've been shown to predict performance pretty well, so it makes sense that over time supervisors would see them as a decent way to hire the right person (although how many of them correlate interview scores with later job performance?).
But I think it's more basic than that. Interviews are the basket that many employers put all their eggs. Why? It's been argued that they're addicted to interviews and stubbornly refuse to acknowledge their flaws. But really I think there are three main things going on with the average hiring supervisor:
1) competing demands for their time/attention;
2) laziness ("let's just do what we did last time"); and
3) a genuine belief that SEEING someone tells you a lot about them. I believe this factor is particularly powerful.
Here's another exercise: ask a bunch of hiring supervisors how many of them would hire someone without interviewing them. I'm guessing no one volunteered.
Why? Why is it so uncomfortable to imagine hiring someone without having seen them? What are they hoping to measure in the interview?
- Job knowledge? Heck, we could put together a 10-15 item m-c test that would do a better job than a couple "what would you do if..." and "tell me about a time when..." questions. The slight additional time involved in putting them together would be worth the increased confidence in the breadth/depth of their knowledge.
- Oral communication? First of all, is that important for the job? Second, is an interview really the highest fidelity situation you can think of? Are you hiring for...interviewee? Ask them to put on a training presentation. Do a role play. Be creative!
- "Something else"? Like, I dunno, friendliness? Social skills? "Fit"? Do you really think you're seeing an accurate picture of these qualities during an interview? Why not use an assessment that's actually designed to measure these things and reference check the heck out of the person?
Aside from the variance you're picking up on job knowledge and communication, really the best you can hope for is that the person messes up. Pity the poor applicant who hasn't memorized the organization's webpage and job bulletin, smiles continuously, and knows which magic words to utter. Instead, what if they:
- Show up to the interview late
- Wear something weird/inappropriate
- Complain inordinately about previous coworkers/bosses
- Ask you bizarre questions
I don't have any data to back this up, but I'd guess nine times out of ten when it comes down to the final group of applicants the "there was just something about them" factor trumps. But can it be quantified? Defended?
So what can be done? There are two big leverage points that organizations need to focus on to avoid the automatic recycling of interviews:
1) HR consultants need to be assertive and available. They should be contacting supervisors when they know about a vacancy, when the advertisement goes out--WHENEVER--just make contact. Find out how you can help. As a hiring supervisor myself, I can tell you it makes a HUGE difference when a consultant asks me if there's anything I can do (to which I respond, "God, yes!").
2) Supervisors--and HR--need to be held accountable for their hires and their reputation as a destination employer. Supervisors, if you have the resources available and you simply fail to take advantage of them, if you never look into why you're having recruiting problems but just keep using the same methods, shame on you. HR, if you know you should be providing this service and you aren't, or if you know you should be better at it but aren't, shame on you.
So it's a big problem, but it's one that with sufficient attention can be tackled. If it seems like I'm being overly negative about interviews and their real-world application, all I can say is...you should see how I feel about resume reviews.
Let me clarify. I don't hate doing interviews, I hate how they're used.
Try this exercise. Spread your arms out as far as they'll go (or envision it). That's how much of a person's competencies employers think they are measuring using the interview.
Now put your hands about a foot apart from each other. That's how much of a person's competencies they're really measuring. And that's if they interview well (which many don't).
So why do employers like interviews so much? Yes, when done right they've been shown to predict performance pretty well, so it makes sense that over time supervisors would see them as a decent way to hire the right person (although how many of them correlate interview scores with later job performance?).
But I think it's more basic than that. Interviews are the basket that many employers put all their eggs. Why? It's been argued that they're addicted to interviews and stubbornly refuse to acknowledge their flaws. But really I think there are three main things going on with the average hiring supervisor:
1) competing demands for their time/attention;
2) laziness ("let's just do what we did last time"); and
3) a genuine belief that SEEING someone tells you a lot about them. I believe this factor is particularly powerful.
Here's another exercise: ask a bunch of hiring supervisors how many of them would hire someone without interviewing them. I'm guessing no one volunteered.
Why? Why is it so uncomfortable to imagine hiring someone without having seen them? What are they hoping to measure in the interview?
- Job knowledge? Heck, we could put together a 10-15 item m-c test that would do a better job than a couple "what would you do if..." and "tell me about a time when..." questions. The slight additional time involved in putting them together would be worth the increased confidence in the breadth/depth of their knowledge.
- Oral communication? First of all, is that important for the job? Second, is an interview really the highest fidelity situation you can think of? Are you hiring for...interviewee? Ask them to put on a training presentation. Do a role play. Be creative!
- "Something else"? Like, I dunno, friendliness? Social skills? "Fit"? Do you really think you're seeing an accurate picture of these qualities during an interview? Why not use an assessment that's actually designed to measure these things and reference check the heck out of the person?
Aside from the variance you're picking up on job knowledge and communication, really the best you can hope for is that the person messes up. Pity the poor applicant who hasn't memorized the organization's webpage and job bulletin, smiles continuously, and knows which magic words to utter. Instead, what if they:
- Show up to the interview late
- Wear something weird/inappropriate
- Complain inordinately about previous coworkers/bosses
- Ask you bizarre questions
I don't have any data to back this up, but I'd guess nine times out of ten when it comes down to the final group of applicants the "there was just something about them" factor trumps. But can it be quantified? Defended?
So what can be done? There are two big leverage points that organizations need to focus on to avoid the automatic recycling of interviews:
1) HR consultants need to be assertive and available. They should be contacting supervisors when they know about a vacancy, when the advertisement goes out--WHENEVER--just make contact. Find out how you can help. As a hiring supervisor myself, I can tell you it makes a HUGE difference when a consultant asks me if there's anything I can do (to which I respond, "God, yes!").
2) Supervisors--and HR--need to be held accountable for their hires and their reputation as a destination employer. Supervisors, if you have the resources available and you simply fail to take advantage of them, if you never look into why you're having recruiting problems but just keep using the same methods, shame on you. HR, if you know you should be providing this service and you aren't, or if you know you should be better at it but aren't, shame on you.
So it's a big problem, but it's one that with sufficient attention can be tackled. If it seems like I'm being overly negative about interviews and their real-world application, all I can say is...you should see how I feel about resume reviews.
Saturday, May 26, 2012
May mega research update
It's time once again for the monthly research round-up. So let's dive right in:
The June International Journal of Selection and Assessment doesn't disappoint; let's take a look:
- More evidence of the link between personality variables and CWBs; this time with concurrent data in China
- Dovetailing nicely with a post I've been working on regarding promotional testing, this research indicates some interesting characteristics of internal test takers
- Why are supervisors open to behavioral interviews but shun discussion of "structure"? Looks like how we communicate about them plays a big role.
- More research on self-efficacy, this time teasing apart the concept a bit.
- Always a popular topic: applicant reactions to selection mechanisms. This time with a sample from Saudi Arabia.
- Speaking of applicant reactions...how about another study? This one comparing U.S. and Vietnamese college students. By the way, not surprisingly work samples came out a winner in both of these studies.
- Next, a fascinating study of a hidden bonus to UIT: despite the cheating element, it likely increases your candidate pool and eventually performance outcomes
- Speaking of response distortion, here's another study, this time of military cadet selection using personality inventories
- Okay, one more on inflation. This time a study of Chinese applicants--no difference compared to American samples.
- Back in March I wrote about a study Jeremy Bernerth published in J.A.P. that got a lot of attention. This time, Bernerth studied ethnic differences and found minority status was negatively related to credit scores.
Moving on to the summer issue of Personnel Psychology:
- The "file drawer problem" is the theory that nonsignificant results are less likely to get published. According to this study, that appears unlikely. But IMHO looking at all correlations is different than looking at the correlations key to one's hypothesis(es)...
- Back to faking (that may be this post's theme!), can response elaboration reduce faking on biodata items? This study suggests so. Although I'm left wondering...what was the impact on validity?
- Speaking of biodata, there are various ways of keying these items. This research suggests the best method depends on your sample size, although rational keying performed the worst.
How about the May issue of Journal of Applied Psychology?
- Well this is interesting...Chad Van Iddekinge and his colleagues have provided an updated meta-analysis on the criterion-related validity of integrity tests. What did they find? Well, the results appear to be less promising than those published previously (e.g., corrected r=.18 for job performance). Much like SIOP's research journal, this time J.A.P. published several commentaries in response to the study that...well, let's just say a debate ensued about the analysis...
- The Dark Triad. It sounds like something in a Dan Brown novel. But in this meta-analysis the authors show that personality characteristics that make up this triad (Machiavellianism, narcissism, and psychopathy) explain some variance in CWBs.
- Why are some people more proactive in seeking career goals than others? It's an important and under-researched question. In this study the authors show that part of the explanation lies in "future work selves", or how people's hopes and aspirations as they relate to work.
- Think self-reports of CWBs are biased? Perhaps not, according to this new study.
- Interested in what causes proactive customer service behavior? According to this multi-national study, self-efficacy is a key (along with service climate).
- Why do some leaders engage in more self-interested behavior than others? Perhaps not surprisingly, it appears due in part to the strength of their moral identity.
The May issue of the Journal of Applied Social Psychology has a couple gems...
- Hey, look, turns out being sensitive to your subordinates pays off. Talk about a lesson that needs frequent repeating...
- And that's it. Oh, wait, just this little study about using Facebook profiles to predict job performance...that I wrote about before....available in FULL right now...
Okay, getting to the end...The May/June issue of HRM:
- An interesting study of adverse impact in promotion decisions for managers in a Fortune 500 retailer. The authors compared three methods (top-down assessment, assessment centers, and multisource appraisal) and the results demonstrate how complex these analyses are!
- Speaking of complex. Think that successful job postings on the web is just fancy graphics? Think again--it still involves some classic factors like the labor market, firm reputation, and compensation incentives. The more things change...
- Identifying future leaders. There are few other issues that are as important for most organizations. Yet how exactly to do it eludes many. These authors propose a model that focuses on four main features: analytical ability, learning agility, drive, and emergent leadership.
Finally, a few from PARE:
- Does item order impact response anxiety? Not according to this study.
- What's that? How do we use a new jacknife procedure to eliminating items and improve structural equation modeling? You're in luck.
- Looks like a lot of research rely on beta weights when interpreting and reporting multiple linear regression results. But there's so much more...
Happy reading!
The June International Journal of Selection and Assessment doesn't disappoint; let's take a look:
- More evidence of the link between personality variables and CWBs; this time with concurrent data in China
- Dovetailing nicely with a post I've been working on regarding promotional testing, this research indicates some interesting characteristics of internal test takers
- Why are supervisors open to behavioral interviews but shun discussion of "structure"? Looks like how we communicate about them plays a big role.
- More research on self-efficacy, this time teasing apart the concept a bit.
- Always a popular topic: applicant reactions to selection mechanisms. This time with a sample from Saudi Arabia.
- Speaking of applicant reactions...how about another study? This one comparing U.S. and Vietnamese college students. By the way, not surprisingly work samples came out a winner in both of these studies.
- Next, a fascinating study of a hidden bonus to UIT: despite the cheating element, it likely increases your candidate pool and eventually performance outcomes
- Speaking of response distortion, here's another study, this time of military cadet selection using personality inventories
- Okay, one more on inflation. This time a study of Chinese applicants--no difference compared to American samples.
- Back in March I wrote about a study Jeremy Bernerth published in J.A.P. that got a lot of attention. This time, Bernerth studied ethnic differences and found minority status was negatively related to credit scores.
Moving on to the summer issue of Personnel Psychology:
- The "file drawer problem" is the theory that nonsignificant results are less likely to get published. According to this study, that appears unlikely. But IMHO looking at all correlations is different than looking at the correlations key to one's hypothesis(es)...
- Back to faking (that may be this post's theme!), can response elaboration reduce faking on biodata items? This study suggests so. Although I'm left wondering...what was the impact on validity?
- Speaking of biodata, there are various ways of keying these items. This research suggests the best method depends on your sample size, although rational keying performed the worst.
How about the May issue of Journal of Applied Psychology?
- Well this is interesting...Chad Van Iddekinge and his colleagues have provided an updated meta-analysis on the criterion-related validity of integrity tests. What did they find? Well, the results appear to be less promising than those published previously (e.g., corrected r=.18 for job performance). Much like SIOP's research journal, this time J.A.P. published several commentaries in response to the study that...well, let's just say a debate ensued about the analysis...
- The Dark Triad. It sounds like something in a Dan Brown novel. But in this meta-analysis the authors show that personality characteristics that make up this triad (Machiavellianism, narcissism, and psychopathy) explain some variance in CWBs.
- Why are some people more proactive in seeking career goals than others? It's an important and under-researched question. In this study the authors show that part of the explanation lies in "future work selves", or how people's hopes and aspirations as they relate to work.
- Think self-reports of CWBs are biased? Perhaps not, according to this new study.
- Interested in what causes proactive customer service behavior? According to this multi-national study, self-efficacy is a key (along with service climate).
- Why do some leaders engage in more self-interested behavior than others? Perhaps not surprisingly, it appears due in part to the strength of their moral identity.
The May issue of the Journal of Applied Social Psychology has a couple gems...
- Hey, look, turns out being sensitive to your subordinates pays off. Talk about a lesson that needs frequent repeating...
- And that's it. Oh, wait, just this little study about using Facebook profiles to predict job performance...that I wrote about before....available in FULL right now...
Okay, getting to the end...The May/June issue of HRM:
- An interesting study of adverse impact in promotion decisions for managers in a Fortune 500 retailer. The authors compared three methods (top-down assessment, assessment centers, and multisource appraisal) and the results demonstrate how complex these analyses are!
- Speaking of complex. Think that successful job postings on the web is just fancy graphics? Think again--it still involves some classic factors like the labor market, firm reputation, and compensation incentives. The more things change...
- Identifying future leaders. There are few other issues that are as important for most organizations. Yet how exactly to do it eludes many. These authors propose a model that focuses on four main features: analytical ability, learning agility, drive, and emergent leadership.
Finally, a few from PARE:
- Does item order impact response anxiety? Not according to this study.
- What's that? How do we use a new jacknife procedure to eliminating items and improve structural equation modeling? You're in luck.
- Looks like a lot of research rely on beta weights when interpreting and reporting multiple linear regression results. But there's so much more...
Happy reading!
Sunday, May 06, 2012
What The Avengers can teach us about high-performance teams
Like millions of other people, I saw Marvel's The Avengers yesterday. It's anticipated to bring in around $200M in its opening weekend. It's full of explosions, space aliens, and people in outlandish costumes.
So what does The Avengers have to do with recruiting and hiring? Nothing. Unless you care about building a high-performance team.
Sure, the movie is nominally about a egomaniacal godlike being, Loki, who is attempting to destroy all humans using an army of aliens summoned through a space portal (I call that Monday). But I think what the movie's really about is the challenge of building and sustaining a high-performance team.
The power of teamwork
Nick Fury has a tough job. As the Director of a secret international espionage and military agency, he's tasked with figuring out how to respond to the threat that Loki poses. While his bosses (let's call them the board) encourage him to use more drastic solutions, Fury steadfastly sticks to his team (that'd be The Avengers--Captain America, Iron Man, Black Widow, Thor, Hawkeye, and The Hulk). Why? Because he believes that their sum exceeds their parts.
Which brings us to our first lessons:
1) Teams are particularly effective when you need the benefits that come with combining talents and skills to deal with a complex situation beyond the abilities of any individual.
2) Successful team leaders believe passionately in the power of the team and are personally devoted to seeing them succeed. They may even need to resist pressure from above and put their reputation on the line.
Dealing with Superstars(heroes)
Fury's biggest challenge lies in assembling the team and keeping them cohesive. Why? Because each one of them is a "high producer". Technically they don't need each other to do great things--and each one of them is extremely confident in their abilities (with Iron Man, played by Robert Downey, Jr. being the poster boy). Each is used to dealing with big challenges themselves, in their own way. Not only are they not used to a "boss" (Fury), but they struggle to form a group identity. Lessons:
3) Recognize that building a team of superstars will be a challenge. And then recognize it publicly. Acknowledge to the team that accomplishing great things will likely not be easy--but it is doable (and in this case, necessary).
4) Plan ahead for what will attract these individual performers to come together. Is it broader recognition? Satisfaction of a job well done? A sense of duty? An opportunity to right wrongs?
5) Prepare for some in-fighting. This is likely inevitable among individuals used to working in their own fashion (and, not coincidentally, getting all the glory).
Bringing them together
So how do you recruit and keep together a group of often-selfish, always stubborn, personalities? Fury gives us some clues...
6) Seek out the diverse talents you need. While each member of The Avengers is super in their own right, each also brings something different. For example, Black Widow is particularly adept at persuasion and interrogation; Captain America is a natural leader; and Thor, well...Thor happens to be related to the main bad guy so he's pretty familiar with the core issues.
7) Appeal to a greater cause. Fury isn't shy about sharing his passion for the idea of The Avengers: the world's most powerful superheroes coming together to defeat evil that threatens the planet. That's a pretty powerful EVP and/or leadership vision, wouldn't you say? (By the way, he's also good at manipulating team emotions for the greater cause)
8) Harness the unique talents of your team members. While Fury is a pretty good recruiter in his own right, he recognizes that certain team members (namely Dr. Bruce Banner a.k.a. The Hulk) might be better wooed by others (i.e., Black Widow). Similarly, as the two scientists, Iron Man and Bruce Banner, are brought together we see immediate results of their complimentary passion and talent.
Keeping them together
Here's where Fury stumbles a little, and it results in the biggest setback the team experiences. The team starts bickering and Fury lets himself get drawn into it. This intensifies the mistrust and distracts them while their enemies infiltrate their headquarters. Only the immediate threat solidifies the team. Last lessons:
9) As a leader, stay above the fray. Teams with strong personalities don't need another one. Your job is to stay and produce calm. Fury would have been better served by calling for a time out so people could cool their jets.
10) Stay focused. Don't let distractions such as momentary setbacks or petty infighting ruin the potential. Remind people why they're there.
11) Be honest. In Fury's case, he's caught with hidden intentions and it drastically lowers team trust. Superstars are often particularly adept at spotting weaknesses in leaders. Don't give them reason to doubt.
There's a lot of research and writing on the topic of building and sustaining high-performance teams. Heck, there are even conferences devoted to it. Interestingly, it's one of the most enduring themes in graphic novels as well (think Fantastic Four, X-Men, etc.). There's a reason why there is so much interest: there are times when special teams are called for, and it's exciting to think about harnessing disparate talents and focusing them on achieving great things.
The best lessons, I think, to draw from The Avengers are that bringing together superstars isn't easy, and keeping them together may be even more difficult. And it's another example of where the strength of leadership can make or break the mission. We may not be trying to recruit and engage superheroes. But we should all be familiar with the challenges inherent in bringing individuals together in the pursuit of a common goal. Even if it isn't saving the world.
So what does The Avengers have to do with recruiting and hiring? Nothing. Unless you care about building a high-performance team.
Sure, the movie is nominally about a egomaniacal godlike being, Loki, who is attempting to destroy all humans using an army of aliens summoned through a space portal (I call that Monday). But I think what the movie's really about is the challenge of building and sustaining a high-performance team.
The power of teamwork
Nick Fury has a tough job. As the Director of a secret international espionage and military agency, he's tasked with figuring out how to respond to the threat that Loki poses. While his bosses (let's call them the board) encourage him to use more drastic solutions, Fury steadfastly sticks to his team (that'd be The Avengers--Captain America, Iron Man, Black Widow, Thor, Hawkeye, and The Hulk). Why? Because he believes that their sum exceeds their parts.
Which brings us to our first lessons:
1) Teams are particularly effective when you need the benefits that come with combining talents and skills to deal with a complex situation beyond the abilities of any individual.
2) Successful team leaders believe passionately in the power of the team and are personally devoted to seeing them succeed. They may even need to resist pressure from above and put their reputation on the line.
Dealing with Superstars(heroes)
Fury's biggest challenge lies in assembling the team and keeping them cohesive. Why? Because each one of them is a "high producer". Technically they don't need each other to do great things--and each one of them is extremely confident in their abilities (with Iron Man, played by Robert Downey, Jr. being the poster boy). Each is used to dealing with big challenges themselves, in their own way. Not only are they not used to a "boss" (Fury), but they struggle to form a group identity. Lessons:
3) Recognize that building a team of superstars will be a challenge. And then recognize it publicly. Acknowledge to the team that accomplishing great things will likely not be easy--but it is doable (and in this case, necessary).
4) Plan ahead for what will attract these individual performers to come together. Is it broader recognition? Satisfaction of a job well done? A sense of duty? An opportunity to right wrongs?
5) Prepare for some in-fighting. This is likely inevitable among individuals used to working in their own fashion (and, not coincidentally, getting all the glory).
Bringing them together
So how do you recruit and keep together a group of often-selfish, always stubborn, personalities? Fury gives us some clues...
6) Seek out the diverse talents you need. While each member of The Avengers is super in their own right, each also brings something different. For example, Black Widow is particularly adept at persuasion and interrogation; Captain America is a natural leader; and Thor, well...Thor happens to be related to the main bad guy so he's pretty familiar with the core issues.
7) Appeal to a greater cause. Fury isn't shy about sharing his passion for the idea of The Avengers: the world's most powerful superheroes coming together to defeat evil that threatens the planet. That's a pretty powerful EVP and/or leadership vision, wouldn't you say? (By the way, he's also good at manipulating team emotions for the greater cause)
8) Harness the unique talents of your team members. While Fury is a pretty good recruiter in his own right, he recognizes that certain team members (namely Dr. Bruce Banner a.k.a. The Hulk) might be better wooed by others (i.e., Black Widow). Similarly, as the two scientists, Iron Man and Bruce Banner, are brought together we see immediate results of their complimentary passion and talent.
Keeping them together
Here's where Fury stumbles a little, and it results in the biggest setback the team experiences. The team starts bickering and Fury lets himself get drawn into it. This intensifies the mistrust and distracts them while their enemies infiltrate their headquarters. Only the immediate threat solidifies the team. Last lessons:
9) As a leader, stay above the fray. Teams with strong personalities don't need another one. Your job is to stay and produce calm. Fury would have been better served by calling for a time out so people could cool their jets.
10) Stay focused. Don't let distractions such as momentary setbacks or petty infighting ruin the potential. Remind people why they're there.
11) Be honest. In Fury's case, he's caught with hidden intentions and it drastically lowers team trust. Superstars are often particularly adept at spotting weaknesses in leaders. Don't give them reason to doubt.
There's a lot of research and writing on the topic of building and sustaining high-performance teams. Heck, there are even conferences devoted to it. Interestingly, it's one of the most enduring themes in graphic novels as well (think Fantastic Four, X-Men, etc.). There's a reason why there is so much interest: there are times when special teams are called for, and it's exciting to think about harnessing disparate talents and focusing them on achieving great things.
The best lessons, I think, to draw from The Avengers are that bringing together superstars isn't easy, and keeping them together may be even more difficult. And it's another example of where the strength of leadership can make or break the mission. We may not be trying to recruit and engage superheroes. But we should all be familiar with the challenges inherent in bringing individuals together in the pursuit of a common goal. Even if it isn't saving the world.
Saturday, April 21, 2012
A fascinating example of an organization making hiring job #1
Recently the employee handbook of Valve, a software and video game development and distribution company (famous for things like Half-Life and it's Steam service) was leaked and frankly it's one of the most interesting things I've read in a long time. It's available several places, including here as of this writing.
Why so interesting? Several reasons...
1) It doesn't look or read like a typical employee handbook. It's very informal, devoid of legalese, balances positivity and expectations, and is graphically very attractive. Not exactly signs of your typical handbook. As an example, the document starts with this statement on the cover:
A fearless adventure in knowing what to do when no one’s there telling you what to do.
(They do keep their formal policies on their Intranet (e.g., about benefits).)
2) It's a very interesting example of how a flat organization describes itself. They don't appear to have much of a management hierarchy so much of the handbook is devoted to explaining how employees are supposed to select assignments, etc.
3) The document itself is editable on their Intranet. Yes, you read that right, the employee handbook is a collaborative document.
Last but definitely not least,
4) This is a great example of how an organization can emphasize that hiring is the most important activity employees engage in. The document is replete with examples. Consider the following passages from the handbook:
p. 6: "hiring is the single most important thing you will ever do at Valve"
p. 14: "We have made significant strides toward bringing more predictability, measurement, and analysis to recruiting. A process that many assume must be treated only as a “soft” art because it has to do with humans, personalities, language, and nuance, actually has ample room for a healthy dose of science."
p. 17: "The thing we work hardest at is hiring good people"
here's my favorite:
p. 44: "Hiring well is the most important thing in the universe. Nothing else comes close. It’s more important than breathing. So when you’re working on hiring—participating in an interview loop or innovating in the general area of recruiting—everything else you could be doing is stupid and should be ignored!"
p. 45: "Missing out on hiring that great person is likely the most expensive kind of mistake we can make...a poor hiring decision can cause lots of damage, and can sometimes go unchecked for too long."
They also talk about how they hire:
p. 47: "When unchecked, people have a tendency to hire others who are lower-powered than themselves...We should hire people more capable than ourselves, not less."
pp. 47-48: "[In some circumstances] hiring someone who is at least capable seems (in the short term) to be smarter than not hiring anyone at all. But that’s actually a huge mistake. We can always bring on temporary/contract help to get us through tough spots, but we should never lower the hiring bar."
There are so many examples it's almost more of a introduction to hiring rather than to the company!
Also, in case you're interested, here's their "work at valve" page, which supports the culture as described in the handbook.
This "leak" has gotten a lot of press, and likely has done a lot to increase its attractiveness as an employer--another reason why this is such a fascinating example.
So I ask you: how much of your organization's handbook discusses the importance of hiring? Do they make it clear it's a shared responsibility of every employee? Is there a passion for hiring right?
Why so interesting? Several reasons...
1) It doesn't look or read like a typical employee handbook. It's very informal, devoid of legalese, balances positivity and expectations, and is graphically very attractive. Not exactly signs of your typical handbook. As an example, the document starts with this statement on the cover:
A fearless adventure in knowing what to do when no one’s there telling you what to do.
(They do keep their formal policies on their Intranet (e.g., about benefits).)
2) It's a very interesting example of how a flat organization describes itself. They don't appear to have much of a management hierarchy so much of the handbook is devoted to explaining how employees are supposed to select assignments, etc.
3) The document itself is editable on their Intranet. Yes, you read that right, the employee handbook is a collaborative document.
Last but definitely not least,
4) This is a great example of how an organization can emphasize that hiring is the most important activity employees engage in. The document is replete with examples. Consider the following passages from the handbook:
p. 6: "hiring is the single most important thing you will ever do at Valve"
p. 14: "We have made significant strides toward bringing more predictability, measurement, and analysis to recruiting. A process that many assume must be treated only as a “soft” art because it has to do with humans, personalities, language, and nuance, actually has ample room for a healthy dose of science."
p. 17: "The thing we work hardest at is hiring good people"
here's my favorite:
p. 44: "Hiring well is the most important thing in the universe. Nothing else comes close. It’s more important than breathing. So when you’re working on hiring—participating in an interview loop or innovating in the general area of recruiting—everything else you could be doing is stupid and should be ignored!"
p. 45: "Missing out on hiring that great person is likely the most expensive kind of mistake we can make...a poor hiring decision can cause lots of damage, and can sometimes go unchecked for too long."
They also talk about how they hire:
p. 47: "When unchecked, people have a tendency to hire others who are lower-powered than themselves...We should hire people more capable than ourselves, not less."
pp. 47-48: "[In some circumstances] hiring someone who is at least capable seems (in the short term) to be smarter than not hiring anyone at all. But that’s actually a huge mistake. We can always bring on temporary/contract help to get us through tough spots, but we should never lower the hiring bar."
There are so many examples it's almost more of a introduction to hiring rather than to the company!
Also, in case you're interested, here's their "work at valve" page, which supports the culture as described in the handbook.
This "leak" has gotten a lot of press, and likely has done a lot to increase its attractiveness as an employer--another reason why this is such a fascinating example.
So I ask you: how much of your organization's handbook discusses the importance of hiring? Do they make it clear it's a shared responsibility of every employee? Is there a passion for hiring right?
Sunday, April 01, 2012
April research update
Okay, so I didn't quite hit my March update...date. But I'm awful close, so without further ado let's take a look at what research has come out lately. And boy is there a lot to cover...
First, the Spring Personnel Psychology, which it should be noted is all free at the time of this writing (!):
- Moore, et al. describe the development and test of a measure of "an individual's propensity to morally disengage", which is really (as the title suggests) about investigating why people do bad things at work. Looks like it has promise beyond existing measures that predict things like CWBs.
- Next, O'Boyle and Aguinis present the results of a fascinating study of the distribution of human performance. Turns out it may not be normal after all, but rather Paretian. This has big implications for...well, all sorts of things, including selection. Read here for more.
- Avery, et al. describe results of a study of racioethnic matching (employees and customers) and the impact on productivity (which turned out to be positive, through customer satisfaction). The authors present several very practical ways of interpreting this finding without jumping to hiring based on race.
Next, the March Journal of Applied Psychology:
- First, van Hooft and Born with a fascinating study of eye-tracking to investigating faking on personality and integrity measures. Looks like eye movement differs depending on the intent to inflate, and it also suggests response time could be an indicator of inflation.
- Next, Madera and Hebl with another eye-tracking study, but this time on the impact that facial stigmatization has on interview performance. Discouragingly (but perhaps not surprising), the results suggest individuals with facial stigmatization may receive lower ratings, in part due to the interviewer being distracted.
- Into core self-evaluations? You might want to read this study by Wu and Griffin, in which they argue that CSEs are predictors of, but also influenced by, contextual factors such as job satisfaction.
- Lievens and Sackett provide evidence that individuals' procedural knowledge of interpersonal behavior may be valuable in predicting performance (in this case, medical students).
- Bernerth and colleagues discuss the usefulness of credit scores in predicting job performance, which I wrote about in an earlier blog post.
Next up, the May Journal of Organizational Behavior:
- Peng, et al. suggest that deployed soldiers with higher levels of conscientiousness and lower levels of neuroticism may be able to better cope with psychological distress (the effect of extraversion was mixed).
- Derous, et al. discuss discrimination in resume screening among individuals who belong to multiple minority groups (in this case with a focus on Arab women). Applicant, job, and recruiter characteristics were all important.
The March Industrial and Organizational Psychology has a fascinating focal article on how I/O psychology and HR can contribute to organizational strategy and competitive advantage. Several commentaries follow, at least one of which directly addresses selection. Move quick, because right now both the focal article and the commentaries are free!
Now here are some miscellaneous articles you may be interested in:
- Jackson, et al. describe a study that suggests going through military training may alter someone's personality...
- Using a 1- or 2-item measure of personality traits, thinking you don't need more than that? Think again.
- Looking for creative thinkers? Gino and Ariely provide evidence that may give you pause: those that are the most creative may be more dishonest...(!)
- Hiring people into a particularly political environment (naw, none of us ever have that)? Chang, et al. present results that suggest you may want to pay attention to their self-monitoring skill and level of conscientiousness...but maybe not in the way you think.
- Still getting over St. Patrick's Day? Then check out van den Born and van Witteloostuijn's research on "shamrock" organizations. They suggest this type of organization may explain the conflicting findings on modern job tenure.
Finally, all the presentations from the 2011 IPAC conference that were previously available only to members have been made public. There is so much good stuff here I can't even begin to summarize it. Just go check it out.
First, the Spring Personnel Psychology, which it should be noted is all free at the time of this writing (!):
- Moore, et al. describe the development and test of a measure of "an individual's propensity to morally disengage", which is really (as the title suggests) about investigating why people do bad things at work. Looks like it has promise beyond existing measures that predict things like CWBs.
- Next, O'Boyle and Aguinis present the results of a fascinating study of the distribution of human performance. Turns out it may not be normal after all, but rather Paretian. This has big implications for...well, all sorts of things, including selection. Read here for more.
- Avery, et al. describe results of a study of racioethnic matching (employees and customers) and the impact on productivity (which turned out to be positive, through customer satisfaction). The authors present several very practical ways of interpreting this finding without jumping to hiring based on race.
Next, the March Journal of Applied Psychology:
- First, van Hooft and Born with a fascinating study of eye-tracking to investigating faking on personality and integrity measures. Looks like eye movement differs depending on the intent to inflate, and it also suggests response time could be an indicator of inflation.
- Next, Madera and Hebl with another eye-tracking study, but this time on the impact that facial stigmatization has on interview performance. Discouragingly (but perhaps not surprising), the results suggest individuals with facial stigmatization may receive lower ratings, in part due to the interviewer being distracted.
- Into core self-evaluations? You might want to read this study by Wu and Griffin, in which they argue that CSEs are predictors of, but also influenced by, contextual factors such as job satisfaction.
- Lievens and Sackett provide evidence that individuals' procedural knowledge of interpersonal behavior may be valuable in predicting performance (in this case, medical students).
- Bernerth and colleagues discuss the usefulness of credit scores in predicting job performance, which I wrote about in an earlier blog post.
Next up, the May Journal of Organizational Behavior:
- Peng, et al. suggest that deployed soldiers with higher levels of conscientiousness and lower levels of neuroticism may be able to better cope with psychological distress (the effect of extraversion was mixed).
- Derous, et al. discuss discrimination in resume screening among individuals who belong to multiple minority groups (in this case with a focus on Arab women). Applicant, job, and recruiter characteristics were all important.
The March Industrial and Organizational Psychology has a fascinating focal article on how I/O psychology and HR can contribute to organizational strategy and competitive advantage. Several commentaries follow, at least one of which directly addresses selection. Move quick, because right now both the focal article and the commentaries are free!
Now here are some miscellaneous articles you may be interested in:
- Jackson, et al. describe a study that suggests going through military training may alter someone's personality...
- Using a 1- or 2-item measure of personality traits, thinking you don't need more than that? Think again.
- Looking for creative thinkers? Gino and Ariely provide evidence that may give you pause: those that are the most creative may be more dishonest...(!)
- Hiring people into a particularly political environment (naw, none of us ever have that)? Chang, et al. present results that suggest you may want to pay attention to their self-monitoring skill and level of conscientiousness...but maybe not in the way you think.
- Still getting over St. Patrick's Day? Then check out van den Born and van Witteloostuijn's research on "shamrock" organizations. They suggest this type of organization may explain the conflicting findings on modern job tenure.
Finally, all the presentations from the 2011 IPAC conference that were previously available only to members have been made public. There is so much good stuff here I can't even begin to summarize it. Just go check it out.
Wednesday, March 28, 2012
Facebook fallout continues
The fallout from earlier reports of employers asking applicants for their Facebook passwords continues. Obviously a nerve was struck.
Today, U.S. Senators Richard Blumenthal and Charles E. Schumer formally asked the Department of Justice (DOJ) and Equal Employment Opportunity Commission (EEOC) to launch an investigation into whether this practice violates federal laws.
From the press release:
"Blumenthal and Schumer argued that this disturbing practice represents a grave intrusion into personal privacy that could set a dangerous precedent for personal privacy and online privacy, make it more difficult for Americans to get jobs, and expose employers to discrimination claims"
"'With few exceptions, employers do not have the need or the right to demand access to applicants’ private, password-protected information.'”
"'In an age where more and more of our personal information – and our private social interactions – are online, it is vital that all individuals be allowed to determine for themselves what personal information they want to make public and protect personal information from their would-be employers. This is especially important during the job-seeking process, when all the power is on one side of the fence.'"
"In their letter to the Justice Department, Blumenthal and Schumer pointed out that two courts have found that when supervisors request employee login credentials, and access otherwise private information with those credentials, that those supervisors may be subject to civil liability. Although those two cases involved current employees, the courts’ reasoning does not clearly distinguish between employees and applicants."
"Blumenthal and Schumer also announced that they are currently drafting legislation that would seek to fill any gaps in federal law that allow employers to require personal login information from prospective employees to be considered for a job."
In related news, on today's Talk of the Nation show, they discussed this issue with a reporter from Wired magazine and an HR consultant.
There are a lot of issues here, ranging from online privacy to public reputation to discrimination, but one that I think deserves more attention is how employers can legitimately get the type of information they're seeking. Again, we're not talking about a background check for, say, a peace officer position, we're talking about your run-of-the-mill clerical job. Basically employers are hungry for any information like displays of poor judgment, a negative attitude about their employer, duplicity in their application, etc.
How might an employer get this type of information without resorting to asking for applicant passwords? It's pretty simple actually, we go back to the basics such as:
1) Reference checks; highly under-used and maligned, with many organizations unaware of technological advances made in this area that make it more likely they'll get the information they need.
2) Work sample/performance tests that simulate actual job tasks. These can be very effective in determining how an applicant will respond in an actual situation (i.e., where things like judgment are important).
3) Situational judgment tests: a lower fidelity version of a performance test that nonetheless can be very effective at assessing candidate's knowledge of, and propensity to engage in, appropriate behavior in various situations.
4) Personality inventories: made to measure things like conscientiousness, openness to experience, and extraversion, which may all be good or bad things depending on the needs of the position.
Bottom line: there are other--better--forms of assessment out there that have been around for a long time and when done well, do the trick. No need to ask for someone's online diary.
As a reminder, for those of you that are IPAC members, I gave a webinar about this topic about a year and a half ago where I gave an overview of the technology as well as a summary of many of the major challenges inherent in this practice. The recording is available in the Members Only area.
Today, U.S. Senators Richard Blumenthal and Charles E. Schumer formally asked the Department of Justice (DOJ) and Equal Employment Opportunity Commission (EEOC) to launch an investigation into whether this practice violates federal laws.
From the press release:
"Blumenthal and Schumer argued that this disturbing practice represents a grave intrusion into personal privacy that could set a dangerous precedent for personal privacy and online privacy, make it more difficult for Americans to get jobs, and expose employers to discrimination claims"
"'With few exceptions, employers do not have the need or the right to demand access to applicants’ private, password-protected information.'”
"'In an age where more and more of our personal information – and our private social interactions – are online, it is vital that all individuals be allowed to determine for themselves what personal information they want to make public and protect personal information from their would-be employers. This is especially important during the job-seeking process, when all the power is on one side of the fence.'"
"In their letter to the Justice Department, Blumenthal and Schumer pointed out that two courts have found that when supervisors request employee login credentials, and access otherwise private information with those credentials, that those supervisors may be subject to civil liability. Although those two cases involved current employees, the courts’ reasoning does not clearly distinguish between employees and applicants."
"Blumenthal and Schumer also announced that they are currently drafting legislation that would seek to fill any gaps in federal law that allow employers to require personal login information from prospective employees to be considered for a job."
In related news, on today's Talk of the Nation show, they discussed this issue with a reporter from Wired magazine and an HR consultant.
There are a lot of issues here, ranging from online privacy to public reputation to discrimination, but one that I think deserves more attention is how employers can legitimately get the type of information they're seeking. Again, we're not talking about a background check for, say, a peace officer position, we're talking about your run-of-the-mill clerical job. Basically employers are hungry for any information like displays of poor judgment, a negative attitude about their employer, duplicity in their application, etc.
How might an employer get this type of information without resorting to asking for applicant passwords? It's pretty simple actually, we go back to the basics such as:
1) Reference checks; highly under-used and maligned, with many organizations unaware of technological advances made in this area that make it more likely they'll get the information they need.
2) Work sample/performance tests that simulate actual job tasks. These can be very effective in determining how an applicant will respond in an actual situation (i.e., where things like judgment are important).
3) Situational judgment tests: a lower fidelity version of a performance test that nonetheless can be very effective at assessing candidate's knowledge of, and propensity to engage in, appropriate behavior in various situations.
4) Personality inventories: made to measure things like conscientiousness, openness to experience, and extraversion, which may all be good or bad things depending on the needs of the position.
Bottom line: there are other--better--forms of assessment out there that have been around for a long time and when done well, do the trick. No need to ask for someone's online diary.
As a reminder, for those of you that are IPAC members, I gave a webinar about this topic about a year and a half ago where I gave an overview of the technology as well as a summary of many of the major challenges inherent in this practice. The recording is available in the Members Only area.
Friday, March 23, 2012
Epic fail: Employers continue asking applicants for Facebook account
The Associated Press put out an article this week about the ongoing trend of employers doing something stupid: asking applicants for their Facebook passwords.
Why stupid? Let me count the ways:
1) It makes the applicants feel like they've applied to a totalitarian regime. And they'll tell others, which goes to your reputation. And what do we know about employer reputation? It drives who applies for your jobs.
2) Employers are likely to see things they wish they hadn't. I don't just mean people passed out drunk at a party, I mean things like religious affiliation.
3) If you're trying to access their profile on your own, many are marked private and you won't see anything.
4) If you ask them to log into their account during the interview, it's like asking to see their personal diary.
5) The content on people's FB page is largely outside their control (e.g., comments, photos they're tagged in).
Oh, and let's not forget:
6) The content of someone's profile--aside from things like education and work history which you should have already--is likely to be totally unrelated to job performance, regardless of its potential usefulness, because frankly most employers aren't graduate students in psychology who have received training on interpreting Big 5 characteristics.
The only caveat I can think of is when this request is made as part of a full background check, in which case pretty much your life is an open book.
Facebook, notoriously unpredictable regarding its privacy policies, subsequently warned employers not to do this...but I don't anticipate that this will stop. Why? Because employers are obsessed (rightfully so) with getting as much--and as varied--information as they possibly can.
This just isn't the right way to do it.
Why stupid? Let me count the ways:
1) It makes the applicants feel like they've applied to a totalitarian regime. And they'll tell others, which goes to your reputation. And what do we know about employer reputation? It drives who applies for your jobs.
2) Employers are likely to see things they wish they hadn't. I don't just mean people passed out drunk at a party, I mean things like religious affiliation.
3) If you're trying to access their profile on your own, many are marked private and you won't see anything.
4) If you ask them to log into their account during the interview, it's like asking to see their personal diary.
5) The content on people's FB page is largely outside their control (e.g., comments, photos they're tagged in).
Oh, and let's not forget:
6) The content of someone's profile--aside from things like education and work history which you should have already--is likely to be totally unrelated to job performance, regardless of its potential usefulness, because frankly most employers aren't graduate students in psychology who have received training on interpreting Big 5 characteristics.
The only caveat I can think of is when this request is made as part of a full background check, in which case pretty much your life is an open book.
Facebook, notoriously unpredictable regarding its privacy policies, subsequently warned employers not to do this...but I don't anticipate that this will stop. Why? Because employers are obsessed (rightfully so) with getting as much--and as varied--information as they possibly can.
This just isn't the right way to do it.
Friday, March 16, 2012
Has your organization given up on rigorous assessment?
I'd like to hear from you:
Has your organization given up on rigorous assessment? Given in to the incessant demands for "faster, cheaper"? Sold its soul to the T&E gods? Failed to replace seasoned, trained, and passionate assessment experts and replaced them with generalists?
Or have you gone the opposite direction--are you innovating and experimenting with new forms of assessment (like the feds are)? Are you selective in whom you choose to work on assessment? Do you resist efforts by management to "dumb down" your selection processes?
Has your organization given up on rigorous assessment? Given in to the incessant demands for "faster, cheaper"? Sold its soul to the T&E gods? Failed to replace seasoned, trained, and passionate assessment experts and replaced them with generalists?
Or have you gone the opposite direction--are you innovating and experimenting with new forms of assessment (like the feds are)? Are you selective in whom you choose to work on assessment? Do you resist efforts by management to "dumb down" your selection processes?
Wednesday, March 14, 2012
Credit scores: Useful for selection, but not for the reason you think
Many employers use credit scores as part of their hiring process, despite the lack of evidence regarding their usefulness, the unpopularity of this practice with applicants, and the fact that the EEOC is not a fan.
Adding to our understanding of the issue, in the March 2012 issue of Journal of Applied Psychology (which I will review fully in my March research update), Jeremy Bernerth and his colleagues describe the results of their study where they found:
1) A significant positive relationship between credit scores and task performance
2) A significant positive relationship between credit scores and OCBs
most likely due to
3) Credit scores being positively related to conscientiousness
because there was
4) No significant relationship between credit scores and workplace deviance, such as theft
Interestingly, credit scores were negatively correlated with agreeableness, which the authors say suggests that more agreeable people are more likely to do things like co-sign on questionable loans.
So the bottom line is credit scores may be valuable because they link to performance ratings and OCBs (likely through personality), not because they predict things like theft.
Which leaves the obvious question: why not just use a personality inventory, which is designed to measure conscientiousness and has little adverse impact (unlike credit scores)?
Adding to our understanding of the issue, in the March 2012 issue of Journal of Applied Psychology (which I will review fully in my March research update), Jeremy Bernerth and his colleagues describe the results of their study where they found:
1) A significant positive relationship between credit scores and task performance
2) A significant positive relationship between credit scores and OCBs
most likely due to
3) Credit scores being positively related to conscientiousness
because there was
4) No significant relationship between credit scores and workplace deviance, such as theft
Interestingly, credit scores were negatively correlated with agreeableness, which the authors say suggests that more agreeable people are more likely to do things like co-sign on questionable loans.
So the bottom line is credit scores may be valuable because they link to performance ratings and OCBs (likely through personality), not because they predict things like theft.
Which leaves the obvious question: why not just use a personality inventory, which is designed to measure conscientiousness and has little adverse impact (unlike credit scores)?
Sunday, February 26, 2012
February 2012 research update: Facebook; hope; 3-option items; cognitive ability and racism
So this month the big buzz is over a research article in JASP on using Facebook profiles to judge personality and predict job performance. So let's tackle that one first:
Kluemper, et al. had three trained university students judge Big Five personality factors (using the IPIP) based on over 200 public Facebook profiles of other students. A much smaller sample (56) was used to determine links between evaluator judgments and job performance as rated by supervisors.
So what did they find? Well, among other things:
(1) inter-rater reliability ranged from .48 to .72 which seems low to me but apparently is typical for other-ratings of personality;
(2) two of the other-ratings (emotional stability and agreeableness) significantly correlated with supervisory ratings (around .30), as did one of the self-ratings (extraversion), suggesting to me the two methods might vary in constructs being measured;
(3) the same two other-ratings added incremental validity beyond self-ratings, whereas the opposite was not true.
To their credit, the authors caution readers (those that actually go beyond the mainstream press articles) about using their results to support hiring decisions based on Facebook. In fact, I'd like to quote them:
"Our findings should not be used by organizations as unbridled support for using SNWs [Social Networking Websites] in employment selection. Without more evidence of criterion-related validity and comparability with established employment selection methods, the use of SNW information for hiring purposes is tenuous. In addition to the potential for employment discrimination, there are privacy rights and ethical issues associated with accessing personal information. Clearly, research investigations of such issues lag current informal HR practices."
So bravo to them for researching this issue, and double-bravo (that's a technical term) for cautioning those with Facebook fever.
Let's move to other research out there...
The March issue of IJSA is chalk-full of great research, so let's take a look:
- Much hay is made over the "type" of validity exhibited by cognitive ability tests (don't get me started). To the extent this distinction makes sense to you, you might enjoy Frank Schmidt's argument that ability tests in certain instances can demonstrate content validity. The article is followed by several commentaries and a response.
- Next up is O'Neill, et al. with a critical review of Stevens and Campion's Teamwork-Knowledge, Skills, and Ability Test.
- Reeder, et al. investigate individual differences as they relate to the perception of cognitive ability tests.
- Now here's somethin': Edwards, et al. argue that the three-option multiple choice item is underutilized. This should be read by everyone who has nightmares about writing distractors.
- Hoffman and Meade argue that score differences across assessment center exercises reflect true differences rather than measurement artifact.
- Got hope? Zysberg shows that, through problem-solving-oriented coping, hope is related to success in a selection process.
We already know that people generally aren't very good at accurately describing their skills and abilities, but in the March Psychological Bulletin, Freund and Kasten provide an illuminating meta-analysis indicating that while the relationship between self-estimated and psychometrically measured cognitive ability is modest (.33), it varies depending upon scales and dimensions.
For anyone measuring contextual performance, consider the relationship between role expectations and OCBs as described by Dierdorff, et al.
Speaking of OCBs, Nielsen et al.'s study suggests when measuring OCB expression in a group setting, consider the level of task interdependence.
On the other hand, if you are interested in job performance ratings, be aware that there may be gender bias, and it differs in direction between performance ratings and promotability ratings (with the latter favoring males), according to a recent meta-analysis by Roth, et al.
Last but definitely not least, research by Hodson and Busseri suggest that individuals lower in cognitive ability may be predisposed to exhibit more racism. This suggests that using ability tests may not only increase the validity of your selection process but lower your chances of discriminatory behavior.
****
On a side/editorial note, I find it fascinating and somewhat frustrating that the research energy still seems to be about teasing out major constructs such as cognitive ability and personality. As a practitioner, I gotta tell ya I'm happy when hiring supervisors use any sort of structured assessment beyond their standard interview. I'd love to see more energy behind increasing the validity of the entire selection process. I doubt I'm the only one that feels that way.
Kluemper, et al. had three trained university students judge Big Five personality factors (using the IPIP) based on over 200 public Facebook profiles of other students. A much smaller sample (56) was used to determine links between evaluator judgments and job performance as rated by supervisors.
So what did they find? Well, among other things:
(1) inter-rater reliability ranged from .48 to .72 which seems low to me but apparently is typical for other-ratings of personality;
(2) two of the other-ratings (emotional stability and agreeableness) significantly correlated with supervisory ratings (around .30), as did one of the self-ratings (extraversion), suggesting to me the two methods might vary in constructs being measured;
(3) the same two other-ratings added incremental validity beyond self-ratings, whereas the opposite was not true.
To their credit, the authors caution readers (those that actually go beyond the mainstream press articles) about using their results to support hiring decisions based on Facebook. In fact, I'd like to quote them:
"Our findings should not be used by organizations as unbridled support for using SNWs [Social Networking Websites] in employment selection. Without more evidence of criterion-related validity and comparability with established employment selection methods, the use of SNW information for hiring purposes is tenuous. In addition to the potential for employment discrimination, there are privacy rights and ethical issues associated with accessing personal information. Clearly, research investigations of such issues lag current informal HR practices."
So bravo to them for researching this issue, and double-bravo (that's a technical term) for cautioning those with Facebook fever.
Let's move to other research out there...
The March issue of IJSA is chalk-full of great research, so let's take a look:
- Much hay is made over the "type" of validity exhibited by cognitive ability tests (don't get me started). To the extent this distinction makes sense to you, you might enjoy Frank Schmidt's argument that ability tests in certain instances can demonstrate content validity. The article is followed by several commentaries and a response.
- Next up is O'Neill, et al. with a critical review of Stevens and Campion's Teamwork-Knowledge, Skills, and Ability Test.
- Reeder, et al. investigate individual differences as they relate to the perception of cognitive ability tests.
- Now here's somethin': Edwards, et al. argue that the three-option multiple choice item is underutilized. This should be read by everyone who has nightmares about writing distractors.
- Hoffman and Meade argue that score differences across assessment center exercises reflect true differences rather than measurement artifact.
- Got hope? Zysberg shows that, through problem-solving-oriented coping, hope is related to success in a selection process.
We already know that people generally aren't very good at accurately describing their skills and abilities, but in the March Psychological Bulletin, Freund and Kasten provide an illuminating meta-analysis indicating that while the relationship between self-estimated and psychometrically measured cognitive ability is modest (.33), it varies depending upon scales and dimensions.
For anyone measuring contextual performance, consider the relationship between role expectations and OCBs as described by Dierdorff, et al.
Speaking of OCBs, Nielsen et al.'s study suggests when measuring OCB expression in a group setting, consider the level of task interdependence.
On the other hand, if you are interested in job performance ratings, be aware that there may be gender bias, and it differs in direction between performance ratings and promotability ratings (with the latter favoring males), according to a recent meta-analysis by Roth, et al.
Last but definitely not least, research by Hodson and Busseri suggest that individuals lower in cognitive ability may be predisposed to exhibit more racism. This suggests that using ability tests may not only increase the validity of your selection process but lower your chances of discriminatory behavior.
****
On a side/editorial note, I find it fascinating and somewhat frustrating that the research energy still seems to be about teasing out major constructs such as cognitive ability and personality. As a practitioner, I gotta tell ya I'm happy when hiring supervisors use any sort of structured assessment beyond their standard interview. I'd love to see more energy behind increasing the validity of the entire selection process. I doubt I'm the only one that feels that way.
Saturday, January 28, 2012
January 2012 research update
Okay, we've got a lot of ground to cover so let's jump right in...
- For you stats fans: Estimating ordinal reliability for Likert-type and ordinal response data
- And more: Effect size: Don't just report it, describe it
- And still more: A confidence interval approach to testing comparability of groups
- Shifting gears: CWBs: There's more than one type
- Femininity and military leadership
- Increasing HR's strategic participation
- HR practices do have an influence on organizational attractiveness
- IQ scores may be influenced by test motivation (in low-stakes settings)
- Learning agility is an important factor in identifying high-potentials
- Slicing and dicing the Big Five
- Extraversion, intelligence, and genetics
- Apparently being agreeable won't make you rich (especially if you're a man)
- Need scientists or engineers? Consider innovativeness and internal LOC
- What if you're looking for leaders of scientists or engineers? Consider three important criteria.
- Increase diversity cues to increase recruitment website stickiness
- Does transformational leadership relate to proactive behavior? Depends on the situation.
- Core self evaluations: More complicated than you might think
- No, seriously, it's much more complicated and involves general versus specific measures
- Speaking of complicated, the relationship between personality traits (Big 5) and performance isn't as simple as many make it out to be
That's all for now! See you in February.
- For you stats fans: Estimating ordinal reliability for Likert-type and ordinal response data
- And more: Effect size: Don't just report it, describe it
- And still more: A confidence interval approach to testing comparability of groups
- Shifting gears: CWBs: There's more than one type
- Femininity and military leadership
- Increasing HR's strategic participation
- HR practices do have an influence on organizational attractiveness
- IQ scores may be influenced by test motivation (in low-stakes settings)
- Learning agility is an important factor in identifying high-potentials
- Slicing and dicing the Big Five
- Extraversion, intelligence, and genetics
- Apparently being agreeable won't make you rich (especially if you're a man)
- Need scientists or engineers? Consider innovativeness and internal LOC
- What if you're looking for leaders of scientists or engineers? Consider three important criteria.
- Increase diversity cues to increase recruitment website stickiness
- Does transformational leadership relate to proactive behavior? Depends on the situation.
- Core self evaluations: More complicated than you might think
- No, seriously, it's much more complicated and involves general versus specific measures
- Speaking of complicated, the relationship between personality traits (Big 5) and performance isn't as simple as many make it out to be
That's all for now! See you in February.
Tuesday, December 27, 2011
Final research update of 2011
Welcome to the last HR Tests research update of 2011! This year has been filled with research developments, building on previous thinking as well as venturing out into new areas. Let's see what the end of the year brings us (hint: a lot)...
We'll start with the Winter 2011 issue of Personnel Psychology:
- If you're looking for executives who excel at strategic thinking, you'll want to pay attention to not only their cognitive ability but their accumulated work history, according to Dragoni, et al.
- I tend to think of realistic job previews (RJPs) as occurring pre-hire, but research by Earnest, et al. suggests that an effective technique is to conduct an oral and written RJP post-hire--and be as honest as possible.
- Meta-analyses are relied upon heavily as summaries of large bodies of research. Roth, et al. point out ways we can make them better, particularly with respect to estimates of validity and group differences.
- One of the downsides of cognitive ability tests is they're not always perceived very well by applicants. Sumanth & Cable show how this perception is influenced by the status of the organization as well as individual status.
- Next, a fascinating description of a scale to measure the idea of calling by Dobrow and Tosti-Kharas. What I found most interesting was how the measure was associated with different criteria for different domains (e.g., art, music, management). You can read a draft version here.
Let's turn now to the January issue of JOB:
- Political skill is a hot topic, and Gentry, et al. demonstrate how the perception of promotability related to this skill varies between bosses, peers, and direct reports. Long story short: it differs, and also depends on behavior (attitude will only take you so far, right?). You can read the full version (at least right now!) here.
- Anyone that's worked with (or has in their family) engineers or scientists knows that they often share some strong traits. This suggests leading these groups that engage in creative activity may require specific attributes, which is what Robledo, et al. set out to describe.
On to the January issue of JOM:
- Core self-evaluation (CSE) is another hot topic, and Chang, et al. provide an important review of 15 years of research, including the meta-analytic support for CSE predicting in-role and extra-role performance.
- Boswell, et al. provide an integrative review of the concept of job search across different situations (e.g., following job loss, while employed).
- One area that deserves more attention is selection into the highest positions within organizations. Withers, et al. provide a review of the process of selecting a director for a board.
Let's not forget v41(2) of Personnel Review:
- Hoglund delves into the topic of talent management as a strategic HRM practice. A fascinating topic that reinforces the importance of HRM as an influence over employee perceptions and behavior.
- The wording of job ads can have important impacts on applicant perceptions and behavior. De Cooman and Pepermans analyze the differences between for-profit and non-profit job ads, and show how only a fraction of the information potentially relevant for job-person match is published.
- Another topic that deserves additional attention is the motivation to expatriate. Altman and Baruch describe results of a qualitative study that may be useful to organizations thinking about attracting and selecting for positions that require this substantial move.
Don't forget this one:
- Johnson, et al. with an important reminder that when looking at the issue of discrimination, using single categories to define groups is probably not the best strategy.
Whew! And last but certainly not least, in the December issue of IOP, Michael McDaniel and colleagues present an argument (similar to ones made elsewhere) that the Uniform Guidelines are outdated and, worse than that, a detriment to the field of selection. The commentaries are many and range from support to passionate disagreement, with a healthy dose of caution (and dare I say...intransigence?) thrown in. Worth a read, particularly for those following the Guidelines literally and those engaged in related litigation. You can read a draft version here.
I hope everyone has a great New Years; here's to a wonderful 2012!
Saturday, December 10, 2011
Forget leadership assessment centers: Use the fat face test
All that time your organization spends on leadership assessment? Throw it out. New research indicates that CEOs with fat faces perform better, so just get out your measuring tape.
Okay, I'm oversimplifying. But this research by Wong and her colleagues is fascinating.
They looked at 55 photos of male CEOs and linked facial width to firm performance. Results? CEOs with higher "facial ratios" (face width relative to face height) led organizations that had "significantly greater firm financial performance."
Why facial width? Previous research has indicated this characteristic is related to aggression and sense of power. People who feel powerful tend to focus more on the big picture rather than little details, and tend to be better at staying on task.
Why men? The relationship between face ratio and behavior has been found to be important only in men. Something to do with testosterone.
But wait, don't throw out your work sample tests quite yet. The researchers also found that the relationship between facial ratio and firm performance was moderated by the style of the leadership team. The relationship was stronger in teams that were "cognitively simple" and saw things in black-or-white terms, apparently due to deference to authority. Not exactly something most organizations dream of. Presumably leadership teams with more nuanced views of the world rely on things beyond what shape their leader's head is.
Yes, I'm being tongue-in-cheek here. But if nothing else, this type of research reinforces the complex and important relationships between genetics, perceptions, and behavior.
Wednesday, November 30, 2011
Bad hires in hindsight
Lately I've been thinking about hires I've seen that went bad. What seemed like the right selection at the time turned into a huge disappointment, or worse--a total nightmare. This is particularly the case when selecting for supervisors and managers.
Looking back at those decisions, what happened? So far I've identified several classic cases:
1) Smallville Syndrome. While the individual may have had success in a smaller role, they were completely unfit to take on the bigger scope of the new job. The strategies that worked before are no longer adequate in the new position. This is similar to thinking your strongest specialist will make the best supervisor, but it can happen even with people that are currently in a similar role.
2) The Loch Ness Monster Effect. On the surface, everything seems fine with the applicant. But lurking below is a monster, just waiting to rear its head. Once in the right position, you will see a side of them you weren't anticipating--and it won't be good.
3) Gumby Complex. A supreme lack of flexibility leads the individual to spurn suggestions to do things differently or be open to new ideas. This often results in a toxic combination of alienating their bosses as well as their subordinates.
4) Prima donna-itis. A hard-charging and competitive individual contributor who consistently performs at the top of their group can yield big returns. But this typically isn't what you're looking for in a supervisor or manager. Instead what can happen is they build their own island where subordinates either conform or get kicked out.
5) Iwannajob condition. The individual wants a job--any job. And they don't care what they do as long as it pays. Except they do. And sooner or later they will realize this, start disengaging, and you've lost valuable training, relationships, and resources.
Turning these around, we see some ideas for improving the selection process:
1) Dig, dig, dig. Resist the urge to hire quickly. Do your homework and gather as much information as you can about potential hires--particularly ones going into leadership positions. Have the courage to not hire anyone if you don't like your candidate pool.
2) Pay attention to the minor details. Something may seem like a minor personality quirk that is overshadowed by the person's strengths. But that quirk may end up making their strengths irrelevant if their new environment causes the quirk to grow into a full-blown condition. Think about how a personality trait might exhibit itself in a new environment.
3) Focus on the job at hand. It's trendy to focus on someone's performance history. And many times this can predict performance in a new job. But when the new position requires new competencies--or a whole new level of them--what happened in the past may not be so instructive. Use a variety of strong assessment devices tied to the position, not the applicants.
4) Consider personality inventories. Yes, they can be challenging to adopt, administer, and interpret. But a lot of organizations use them successfully to help get underneath the shiny exterior. Just be very careful when selecting the particular tool to use.
5) When hiring for a supervisor or manager position, spend 80% of your time looking at personality and communication style, 20% on technical competencies. People want to be led, motivated, and engaged, not micromanaged by someone who feels like they can do the job better than you can.
6) Try to find out why the person wants the job. This isn't easy, and generally won't be had by simply asking, "So...why do you want this job?" You can start with that question, but follow it up with a bunch more that get at whether the person has really thought about their fit and what they hope to accomplish in the position.
This is of course just a sample of what can go wrong and some quick suggestions. As those of you that have been doing this a while know, hiring the right person is usually not quick and easy--particularly for leadership positions. But boy is the effort spent up front worth it. Just ask anyone who's ever had a bad boss.
Sunday, November 20, 2011
Research update #583: Impression management and a lot more
Okay, I've got a lot of ground to cover this time, so buckle up...
Let's start with the December issue of IJSA:
- Looks like how much applicants try to make themselves look good varies by country
- Is applicant faking behavior related to job performance? Kinda depends on your definitions.
- Research has found that emotional intelligence can be related to work attitudes. This appears to be in part because of an increased situational judgment effectiveness.
- Speaking of situational judgment...in terms of job knowledge, knowing what to do is different than knowing what not to do
- What impact does a resume have on a recruiter? Depends on what assumptions they make about you after reading it.
- How to people select--and continue with--an executive coach? By looking at things like their ability to forge a partnership.
- How do Canadian firms do in terms of using tests other than interviews? Not so well, it turns out.
Let's move to the October issue of JASP, where there's just one article but it's a good one. Researchers continued the (depressing) finding that applicant names impact pre-interview impressions. Specifically, the more a name was Anglicized, the more favorable the impression was when hiring for an outside sales job.
Next comes the November issue of JAP:
- A new meta-analysis of the FFM of personality and its relationship to OCBs and task performance.
- Measures of interest haven't gotten a lot of love as selection devices. Looks like we need to tease out the constructs a little because they could be more helpful than we thought.
- Applicants trying to create a certain image during an interview are better off doing this after an initial flub or relying solely on self-promotion rather than making up an image.
A few from the November issue of JPSP:
- Another on impression management (not selection-specific) that goes into more detail about the topic (e.g., how many tactics people use, their accuracy)
- A caution about using the Revised NEO-PI in different cultures due to DIF.
Next, a call for more transparency in false-positive findings.
Last but not least, those of you interested in the potential of social ratings of performance being used for selection might be interested in this study of RateMyProfessors.com, which found student ratings are likely to be useful measures of teacher quality.
Let's start with the December issue of IJSA:
- Looks like how much applicants try to make themselves look good varies by country
- Is applicant faking behavior related to job performance? Kinda depends on your definitions.
- Research has found that emotional intelligence can be related to work attitudes. This appears to be in part because of an increased situational judgment effectiveness.
- Speaking of situational judgment...in terms of job knowledge, knowing what to do is different than knowing what not to do
- What impact does a resume have on a recruiter? Depends on what assumptions they make about you after reading it.
- How to people select--and continue with--an executive coach? By looking at things like their ability to forge a partnership.
- How do Canadian firms do in terms of using tests other than interviews? Not so well, it turns out.
Let's move to the October issue of JASP, where there's just one article but it's a good one. Researchers continued the (depressing) finding that applicant names impact pre-interview impressions. Specifically, the more a name was Anglicized, the more favorable the impression was when hiring for an outside sales job.
Next comes the November issue of JAP:
- A new meta-analysis of the FFM of personality and its relationship to OCBs and task performance.
- Measures of interest haven't gotten a lot of love as selection devices. Looks like we need to tease out the constructs a little because they could be more helpful than we thought.
- Applicants trying to create a certain image during an interview are better off doing this after an initial flub or relying solely on self-promotion rather than making up an image.
A few from the November issue of JPSP:
- Another on impression management (not selection-specific) that goes into more detail about the topic (e.g., how many tactics people use, their accuracy)
- A caution about using the Revised NEO-PI in different cultures due to DIF.
Next, a call for more transparency in false-positive findings.
Last but not least, those of you interested in the potential of social ratings of performance being used for selection might be interested in this study of RateMyProfessors.com, which found student ratings are likely to be useful measures of teacher quality.
Sunday, November 06, 2011
How important is assessment, really?
Prepare for a little blasphemy.
Over the last few months my job--and focus--has changed dramatically. Historically I've been a "testing" guy. Question about job analysis? Item writing? I'm there.
Then, a few years ago, I started managing a team that did a more than assessment--a lot more. In fact, even though assessment is in their job description, the team spends most of their time counseling supervisors on performance management. Of course some of this is because the testing workload is down, but it's also a function of demand for advice in this area.
In July we found out our department's budget was being cut; to the tune of about 10%. We adjusted and tightened our belts, but in the end it wasn't enough and we had to plan for layoffs. I was recruited to be one of the coordinators of said layoff, and thus began the dramatic work shift.
That's all a really long way of saying that my focus lately has not been on recruitment and hiring. I've been thinking a lot more about what keeps people going in difficult times. Sure, the KSAOs they bring to the table are important, but other things raise in importance during times of uncertainty and lack of control.
Which got me to thinking: how important IS assessment really? Even at our best, we can predict only about a third of individual job performance. What's going on with that other 2/3rds?
You're probably familiar with models of job performance, so I won't bore you. Suffice to say that a lot goes into job performance. So that person you hired that aced your assessments? Not guaranteed to be super star. If they end up being supervised by an incompetent manager, their inner greatness may never reach the surface. If they have a death in the family, you better believe their focus is not going to be on work for a while and job performance may not be at maximum.
Let's think about job performance as a pie. Top-notch assessment can predict about a third of that pie. What else is in that pie--and more importantly, how big are the slices? Things like:
- motivation
- supervision style
- role clarity
- co-worker support
- mood
- resources
- performance feedback
- stress
(I tried with no luck to track down a comprehensive path model, maybe one of you can point one out)
Now we can't control all of these things (although as HR professionals we certainly can consult on a lot of them--clear duty statements, supervisor training and accountability, engagement surveys, etc.), but what we CAN do is take the rigor we bring to the study of assessment and apply it to other aspects related to job performance. If you look at HR research outside of assessment I think you'll find that the level of analysis is, shall we say, sometimes lacking.
Don't get me wrong: assessment will always be important. The legal rationale is IMHO the least compelling. Instead, there is proven, substantial, utility in implementing best practices for employee assessment.
But lately I can't help but thinking: Are we spending too much time thinking about--and studying--how we bring people in to an organization, and not enough time thinking about what happens once they get there?
Sunday, October 16, 2011
Advice to bballstud_23@mail.com: Apply using a different email address
It's not that I don't appreciate your desire to be different, but I worry about your chances of getting a job.
Why? Because if you apply for a job using that email address, you're going to look:
a) anti-social
b) unprofessional
c) careless
d) inexperienced
This may seem obvious, but according to a little gem tucked away in the latest TIP by Sachau and colleagues, not only are people with, shall we say, "casual" email addressed perceived negatively, they actually score lower on preemployment tests.
The authors used data from over 15,000 actual job applicants who had completed various tests from SHL. Email addresses were coded on a variety of scales such as craziness/insanity, drugs/alcohol, and sci-fi/geeky/nerdy. Then they analyzed the differences in test scores between people with more professional emails with the more casual ones.
Not only did raters agree on the lack of professionalism these types of addresses exhibited, individuals with these addresses actually performed lower on measures of conscientiousness, professionalism, work experience, and total score.
And while the effect sizes weren't huge (there was about a 10% difference in group means between high and low groups), it should give you pause that even if your address is questionable as opposed to outright silly, you will be perceived as unprofessional and may be assumed to have lower test scores.
...unless that's what you're going for.
Saturday, September 24, 2011
Research update + happy anniversary...to me!
Two things this time: we've got a lot of research to go over, and then a bit of a celebration! First the research.
The September issue of the Journal of Applied Psychology is out. Let's see what it has to offer:
- Using performance ratings as an assessment or a criteria? You'll want to look at Ng, et al.'s study of leniency and halo errors among superiors, peers, and subordinates of a sample of military officers.
- Speaking of criteria, you may be interested in Northcraft et al.'s study of how characteristics of the feedback environment influence resource use among competing tasks. Interesting stuff.
- Okay, let's turn to something more traditional. Berry, et al. look at correlations between cognitive ability tests and performance among different ethnic groups. Not surprising to those of you familiar with the research, the largest difference found was between White and Black samples.
- Another traditional (but always interesting) topic: designing Pareto-optimal selection systems when applicants belong to a mixture of populations. Check out De Corte, et al.'s piece. Oh, you might be interested in the in-press version.
- Dr. Lievens (a co-author on the previous study) has been busy. He and Fiona Patterson collaborate on a study of the incremental validity of simulations, both low fidelity (SJTs in this case) and high fidelity (assessment centers), beyond knowledge tests. Yes, both had incremental validity, and interestingly ACs showed incremental validity beyond SJTs. Check out the in press version as well.
- Wondering whether re-testing degrades criterion-related validity or impacts group differences? You're in luck because Van Iddekinge, et al. present the results of a study of just that. Short version? Re-testing actually did a lot of good.
- I know what you're thinking: "Might Lancaster's mid-P correction to Fischer's exact test improve adverse impact analysis?" Check out Biddle & Morris' study for an answer.
- And now that you've had your fill of that statistical analysis, you find your mind wandering to effect size indices for analyzing measurement equivalence. I'm right there with ya. So are Nye & Drasgow.
Let's turn now to the October issue of the Journal of Personality and Social Psychology because there are a few articles I think might interest you...
- First, Lara Kammrath with a fascinating study of how people's understanding of trait behaviors influence their anticipation of how others will react.
- Speaking of fascinating, George, et al. present the results of a 50-year longitudinal study of personality traits predicting career behavior and success among women. It makes you realize again how much has changed since the 1960s!
- I tell ya, this issue is chalk full of goodness. Carlson et al. demonstrate that people can make valid distinctions between how they see themselves and how others see them--potentially informing the debate on personality inventories.
- Lastly, a piece by Specht et al. on how personality changes over the life span and why this might be. Fascinating implications for using personality inventories for selection.
Bonus article: remember how I mentioned using performance ratings above? Well you might be interested in an article by Lynn & Sturman in the most recent Journal of Applied Social Psychology where they found that restaurant customers sometimes rated the performance of same-race servers as better than those of different races--but it depended on the criterion.
FINALLY, I'm proud to announce that this blog has officially been going strong for five years. My first post (now incredibly hard to read) was in September of 2006. Back then the only other similar blog was Jamie Madigan's great (but now sadly defunct) blog, Selection Matters. My first email subscriber (from DDI if you're curious) came on a month later. Now I have almost 150 email subscribers and at least a couple hundred more who follow the feed. Around 3,000 individuals visit the site each month from over a hundred countries/territories (U.S., India, and Canada are 1-2-3). It's a labor of love and I thank you for reading!
The September issue of the Journal of Applied Psychology is out. Let's see what it has to offer:
- Using performance ratings as an assessment or a criteria? You'll want to look at Ng, et al.'s study of leniency and halo errors among superiors, peers, and subordinates of a sample of military officers.
- Speaking of criteria, you may be interested in Northcraft et al.'s study of how characteristics of the feedback environment influence resource use among competing tasks. Interesting stuff.
- Okay, let's turn to something more traditional. Berry, et al. look at correlations between cognitive ability tests and performance among different ethnic groups. Not surprising to those of you familiar with the research, the largest difference found was between White and Black samples.
- Another traditional (but always interesting) topic: designing Pareto-optimal selection systems when applicants belong to a mixture of populations. Check out De Corte, et al.'s piece. Oh, you might be interested in the in-press version.
- Dr. Lievens (a co-author on the previous study) has been busy. He and Fiona Patterson collaborate on a study of the incremental validity of simulations, both low fidelity (SJTs in this case) and high fidelity (assessment centers), beyond knowledge tests. Yes, both had incremental validity, and interestingly ACs showed incremental validity beyond SJTs. Check out the in press version as well.
- Wondering whether re-testing degrades criterion-related validity or impacts group differences? You're in luck because Van Iddekinge, et al. present the results of a study of just that. Short version? Re-testing actually did a lot of good.
- I know what you're thinking: "Might Lancaster's mid-P correction to Fischer's exact test improve adverse impact analysis?" Check out Biddle & Morris' study for an answer.
- And now that you've had your fill of that statistical analysis, you find your mind wandering to effect size indices for analyzing measurement equivalence. I'm right there with ya. So are Nye & Drasgow.
Let's turn now to the October issue of the Journal of Personality and Social Psychology because there are a few articles I think might interest you...
- First, Lara Kammrath with a fascinating study of how people's understanding of trait behaviors influence their anticipation of how others will react.
- Speaking of fascinating, George, et al. present the results of a 50-year longitudinal study of personality traits predicting career behavior and success among women. It makes you realize again how much has changed since the 1960s!
- I tell ya, this issue is chalk full of goodness. Carlson et al. demonstrate that people can make valid distinctions between how they see themselves and how others see them--potentially informing the debate on personality inventories.
- Lastly, a piece by Specht et al. on how personality changes over the life span and why this might be. Fascinating implications for using personality inventories for selection.
Bonus article: remember how I mentioned using performance ratings above? Well you might be interested in an article by Lynn & Sturman in the most recent Journal of Applied Social Psychology where they found that restaurant customers sometimes rated the performance of same-race servers as better than those of different races--but it depended on the criterion.
FINALLY, I'm proud to announce that this blog has officially been going strong for five years. My first post (now incredibly hard to read) was in September of 2006. Back then the only other similar blog was Jamie Madigan's great (but now sadly defunct) blog, Selection Matters. My first email subscriber (from DDI if you're curious) came on a month later. Now I have almost 150 email subscribers and at least a couple hundred more who follow the feed. Around 3,000 individuals visit the site each month from over a hundred countries/territories (U.S., India, and Canada are 1-2-3). It's a labor of love and I thank you for reading!
Subscribe to:
Posts (Atom)