Several research updates this time, including fascinating studies of how computers can assist with interview training, and how a brief writing exercise can lower stereotype threat for women:
International Journal of Selection and Assessment, December, 2016:
- International support for the cultural intelligence scale
- Looking to improve applicant interview performance? Maybe a computer can help.
- This study found that time lag and g-loading are important factors impacting re-testing results
- Status-seeking seems to be an important individual difference when looking at self-presentation behaviors, including exaggeration and faking in job search
- Development and validation of a 360-degree measure of leadership personality
Personnel Psychology, Winter 2016:
- Do CEOs significantly impact firm performance? This study found evidence that they do.
- More evidence that the assumption that performance is normally distributed is questionable
- A more accurate correction for range restriction is presented, and an example analysis indicates the relationship between the Big 5 and job satisfaction may be greater than previously believed
Journal of Organizational Behavior, November, 2016 (which includes several articles devoted to the importance of theory in organizational sciences):
- Are self-focused or other-focused recruiting advertisements more effective? This study suggests it depends not only on cultural differences but individual regulatory focus
Journal of Applied Psychology, October, 2016:
- A suggestion for improving meta-analytic structural equation modeling
Journal of Applied Psychology, November, 2016:
- A fascinating study of how having women compose a brief written description of their personal values can help ameliorate stereotype threat in competitive environments
Showing posts with label Interviews. Show all posts
Showing posts with label Interviews. Show all posts
Tuesday, November 15, 2016
Saturday, June 11, 2016
One way to reduce interviewer leniency/severity
A persistent challenge in interviews is that certain interviewers tend to be lenient (i.e., score candidates highly) while others are consistently more severe (i.e., score candidates lower). This of course is not ideal as it introduces measurement bias as well as reduces the defensibly of the process.
One way to reduce these tendencies discussed by Hartwell and Campion in the June 2016 issue of Journal of Applied Psychology is to provide interviewers with what they call "normative feedback interventions." Basically what this means is giving interviewers data on how they rate candidates over time compared to how other interviewers rated. It can reveal to interviewers that they tend to rate candidates more harshly, or more easily, than others.
What Hartwell and Campion found in their study (of over 20,000 interviews using more than 100 interviewers) is that by providing this feedback to interviewers, it minimized interviewer differences and increased interview reliability--both obviously good things in terms of quality of the process. Interestingly, it did not seem to impact the validity of the interviews, but it did impact which particular candidates were hired.
Up until now, one of the most often recommended practices for reducing rating errors has been pre-interview instructions and guidance regarding these errors. What this study suggests is we can do even better by providing interviewers with objective data about their ratings over time. Listening to someone talking about rating bias probably feels a lot different than actually seeing how you do compared to your peers!
Wednesday, May 04, 2016
Research update
A few new journal issues have come out lately:
Summer 2016 Personnel Psychology, including:
Transparency of Assessment Centers: Low Criterion-related Validity but Greater Opportunity to Perform?
May 2016 Journal of Applied Psychology, including:
Initial impressions: What they are, what they are not, and how they influence structured interview outcomes.
Racioethnicity, community makeup, and potential employees’ reactions to organizational diversity management approaches.
June 2016 International Journal of Selection and Assessment, including:
Applicant Reactions to Selection Events: Four studies into the role of attributional style and fairness perceptions
Behavioral Cues as Indicators of Deception in Structured Employment Interviews
The Role of Self-focused Attention and Negative Self-thought in Interview Anxiety: A test of two interventions
The Influence of Candidate Social Effectiveness on Assessment Center Performance Ratings: A field study
Discrimination due to Ethnicity and Gender: How susceptible are video-based job interviews?
A Comparison of General and Work-specific Personality Measures as Predictors of Organizational Citizenship Behavior
The Perceived Nature and Incidence of Dysfunctional Assessment Center Features and Processes
Who is Being Judged Promotable: Good actors, high performers, highly committed or birds of a feather?
Summer 2016 Personnel Psychology, including:
Transparency of Assessment Centers: Low Criterion-related Validity but Greater Opportunity to Perform?
May 2016 Journal of Applied Psychology, including:
Initial impressions: What they are, what they are not, and how they influence structured interview outcomes.
Racioethnicity, community makeup, and potential employees’ reactions to organizational diversity management approaches.
June 2016 International Journal of Selection and Assessment, including:
Applicant Reactions to Selection Events: Four studies into the role of attributional style and fairness perceptions
Behavioral Cues as Indicators of Deception in Structured Employment Interviews
The Role of Self-focused Attention and Negative Self-thought in Interview Anxiety: A test of two interventions
The Influence of Candidate Social Effectiveness on Assessment Center Performance Ratings: A field study
Discrimination due to Ethnicity and Gender: How susceptible are video-based job interviews?
A Comparison of General and Work-specific Personality Measures as Predictors of Organizational Citizenship Behavior
The Perceived Nature and Incidence of Dysfunctional Assessment Center Features and Processes
Who is Being Judged Promotable: Good actors, high performers, highly committed or birds of a feather?
Sunday, February 28, 2016
New journal issues
Two new journal issues to make you aware of:
International Journal of Selection and Assessment - March 2016
Unintended Consequences of Transparency During Personnel Selection: Benefitting some candidates, but harming others?
Ethnic Differences in Perceptions of Cognitive Ability Tests: The explanatory role of self-serving attributions
Conditional Reasoning Test for Aggression: Further evidence about incremental validity
For Love or for Money: Intrinsic and extrinsic value congruence in recruitment
Social Influences in Recruitment: When is word-of-mouth most effective?
Highlighting Tensions in Recruitment and Selection Research and Practice
Tests of Integrity, HEXACO Personality, and General Mental Ability, as Predictors of Integrity Ratings in the Royal Dutch Military Police
Training Affects Variability in Training Performance Both Within and Across Jobs
Examining Applicant Reactions to Different Media Types in Character-based Simulations for Employee Selection
When Will Interviewers Be Willing to Use High-structured Job Interviews? The role of personality
Journal of Applied Psychology - March 2016
How and why do interviewers try to make impressions on applicants? A qualitative study.
The long road to employment: Incivility experienced by job seekers.
The role of self-determined motivation in job search: A dynamic approach.
International Journal of Selection and Assessment - March 2016
Unintended Consequences of Transparency During Personnel Selection: Benefitting some candidates, but harming others?
Ethnic Differences in Perceptions of Cognitive Ability Tests: The explanatory role of self-serving attributions
Conditional Reasoning Test for Aggression: Further evidence about incremental validity
For Love or for Money: Intrinsic and extrinsic value congruence in recruitment
Social Influences in Recruitment: When is word-of-mouth most effective?
Highlighting Tensions in Recruitment and Selection Research and Practice
Tests of Integrity, HEXACO Personality, and General Mental Ability, as Predictors of Integrity Ratings in the Royal Dutch Military Police
Training Affects Variability in Training Performance Both Within and Across Jobs
Examining Applicant Reactions to Different Media Types in Character-based Simulations for Employee Selection
When Will Interviewers Be Willing to Use High-structured Job Interviews? The role of personality
Journal of Applied Psychology - March 2016
How and why do interviewers try to make impressions on applicants? A qualitative study.
The long road to employment: Incivility experienced by job seekers.
The role of self-determined motivation in job search: A dynamic approach.
Thursday, February 12, 2015
"Power posing" impacts interview performance
Many people have watched Amy Cuddy's riveting TED talk on how body language impacts thought processes. Specifically, over 23.5 million people as of today have watched that particular video, millions more through YouTube and other outlets.
In the presentation she specifically talks about how "power posing"--standing in a way that denotes confidence (hands on hips)--impacts testosterone and cortisol levels and ultimately behavior.
In an upcoming issue of the Journal of Applied Psychology, Cuddy and her colleagues describe the results of an experiment where university students were told to adopt either a high-power (shown on left) or low-power (shown on right) posture:
They were instructed to stand in this pose for 5-6 minutes while preparing for a job interview speech. They then delivered said speech, which was videotaped, to two evaluators/interviewers. These evaluators subsequently rated the participants on (a) performance, and (b) hirability. They also judged the participants on their verbal and nonverbal behavior.
Results (N=61)? Those who prepared for their speech while in a high-power pose were rated significantly higher (p<.01) on both factors than those who prepared in a low-power pose. And the results could not be explained by nonverbal behavior in the interview itself. Per the authors, "Compared to low-power posers, high-power posers appeared to better maintain their composure, project more confidence, and present more captivating and enthusiastic speeches, which led to higher overall performance evaluations."
This suggest, in the words of the authors, "By nonverbally manipulating their own sense of power, the high-power posers were effectively imbued with the psychological and physiological advantages typically associated with high power, despite their low-power position relative to the evaluators."
So what does this mean? I believe it has several important implications. First, it indicates a potential source of "error", akin to test anxiety, that may impact assessment performance. Second, it suggests a potential avenue that those plagued by test anxiety may pursue to increase their chances of success. This includes those who experience stereotype threat. Third, it may help explain why even a "perfect" measurement of job-related KSAs does not yield a perfect correlation with performance (I'm thinking of similar confounding effects like mood or physical appearance).
You can read what appears to be the submitted version here. I recommend reading it as well as the excellent TED talk if this subject interests you.
In the presentation she specifically talks about how "power posing"--standing in a way that denotes confidence (hands on hips)--impacts testosterone and cortisol levels and ultimately behavior.
In an upcoming issue of the Journal of Applied Psychology, Cuddy and her colleagues describe the results of an experiment where university students were told to adopt either a high-power (shown on left) or low-power (shown on right) posture:
They were instructed to stand in this pose for 5-6 minutes while preparing for a job interview speech. They then delivered said speech, which was videotaped, to two evaluators/interviewers. These evaluators subsequently rated the participants on (a) performance, and (b) hirability. They also judged the participants on their verbal and nonverbal behavior.
Results (N=61)? Those who prepared for their speech while in a high-power pose were rated significantly higher (p<.01) on both factors than those who prepared in a low-power pose. And the results could not be explained by nonverbal behavior in the interview itself. Per the authors, "Compared to low-power posers, high-power posers appeared to better maintain their composure, project more confidence, and present more captivating and enthusiastic speeches, which led to higher overall performance evaluations."
This suggest, in the words of the authors, "By nonverbally manipulating their own sense of power, the high-power posers were effectively imbued with the psychological and physiological advantages typically associated with high power, despite their low-power position relative to the evaluators."
So what does this mean? I believe it has several important implications. First, it indicates a potential source of "error", akin to test anxiety, that may impact assessment performance. Second, it suggests a potential avenue that those plagued by test anxiety may pursue to increase their chances of success. This includes those who experience stereotype threat. Third, it may help explain why even a "perfect" measurement of job-related KSAs does not yield a perfect correlation with performance (I'm thinking of similar confounding effects like mood or physical appearance).
You can read what appears to be the submitted version here. I recommend reading it as well as the excellent TED talk if this subject interests you.
Wednesday, August 06, 2014
Research update
I can't believe it's been three months since a research update. I was waiting until I got critical mass, and with the release of the September issues of IJSA, I think I've hit it.
So let's start there:
- Experimenting with using different rating scales on SJTs (with "best and worst" response format doing the best of the traditional scales)
- Aspects of a semi-structured interview added incremental validity over cognitive ability in predicting training performance
- Studying the use of preselection methods (e.g., work experience) prior to assessment centers in German companies
- The proposed general factor of personality may be useful in selection contexts (this one was a military setting)
- Evidence that effective leaders show creativity and political skill
- Investigating the relationship (using survey data) between personality facets and CWBs (with emotional stability playing a key role)
- Corrections for indirect range restriction boosted the upper end of structured interview validity substantially
- A method of increasing the precision of simulations that analyze group mean differences and adverse impact
- A very useful study that looked at the prediction of voluntary turnover as well as performance using biodata and other applicant information, including recruitment source, among a sample of call center applicants. Reuslts? Individuals who had previously applied, chose to submit additional information, were employed, or were referrals had significantly less voluntary turnover.
Moving on...let's check out the May issue of JAP; there are only two articles but both worth looking at:
- First, a fascinating study of the firm-level impact of effective staffing and training, suggesting that the former allow organizations greater flexibility and adaptability (e.g., to changing financial conditions).
- Second, another study of SJT response formats. The researchers found, using a very large sample, the "rate" format (e.g., "rate each of the following options in terms of effectiveness") to be superior in terms of validity, reliability, and group differences.
Next, the July issue of JOB, which is devoted to leadership:
- You might want to check out this overview/critique of the various leadership theories.
- This study suggests that newer models proposing morality as an important component of leadership success have methodological flaws.
- Last, a study of why Whites oppose affirmative action programs
Let's move to the September issue of Industrial and Organizational Psychology:
- The first focal article discusses the increasing movement of I/O psychology to business schools. The authors found evidence that this is due in large part to some of the most active and influential I/O researchers moving to business schools.
- The second is about stereotype threat--specifically its importance as a psychological construct and the paucity of applied research about it.
Coming into the home stretch, the Summer issue of Personnel Psych:
- The distribution of individual performance may not be normal if, as these researchers suggest, "star performers" have emerged
- Executives with high levels of conscientiousness and who display transformational leadership behavior may directly contribute to organizational performance
So let's start there:
- Experimenting with using different rating scales on SJTs (with "best and worst" response format doing the best of the traditional scales)
- Aspects of a semi-structured interview added incremental validity over cognitive ability in predicting training performance
- Studying the use of preselection methods (e.g., work experience) prior to assessment centers in German companies
- The proposed general factor of personality may be useful in selection contexts (this one was a military setting)
- Evidence that effective leaders show creativity and political skill
- Investigating the relationship (using survey data) between personality facets and CWBs (with emotional stability playing a key role)
- Corrections for indirect range restriction boosted the upper end of structured interview validity substantially
- A method of increasing the precision of simulations that analyze group mean differences and adverse impact
- A very useful study that looked at the prediction of voluntary turnover as well as performance using biodata and other applicant information, including recruitment source, among a sample of call center applicants. Reuslts? Individuals who had previously applied, chose to submit additional information, were employed, or were referrals had significantly less voluntary turnover.
Moving on...let's check out the May issue of JAP; there are only two articles but both worth looking at:
- First, a fascinating study of the firm-level impact of effective staffing and training, suggesting that the former allow organizations greater flexibility and adaptability (e.g., to changing financial conditions).
- Second, another study of SJT response formats. The researchers found, using a very large sample, the "rate" format (e.g., "rate each of the following options in terms of effectiveness") to be superior in terms of validity, reliability, and group differences.
Next, the July issue of JOB, which is devoted to leadership:
- You might want to check out this overview/critique of the various leadership theories.
- This study suggests that newer models proposing morality as an important component of leadership success have methodological flaws.
- Last, a study of why Whites oppose affirmative action programs
Let's move to the September issue of Industrial and Organizational Psychology:
- The first focal article discusses the increasing movement of I/O psychology to business schools. The authors found evidence that this is due in large part to some of the most active and influential I/O researchers moving to business schools.
- The second is about stereotype threat--specifically its importance as a psychological construct and the paucity of applied research about it.
Coming into the home stretch, the Summer issue of Personnel Psych:
- The distribution of individual performance may not be normal if, as these researchers suggest, "star performers" have emerged
- Executives with high levels of conscientiousness and who display transformational leadership behavior may directly contribute to organizational performance
Rounding out my review, check out a few recent articles from PARE:
- I'm not even gonna attempt to summarize this, so here's the title: Multiple-Group confirmatory factor analysis in R – A tutorial in measurement invariance with continuous and ordinal indicators
- Improving exploratory factor analysis for ordinal data
- Improving multidimensional adaptive testing
Last but not least, it's not related to recruitment or assessment, but check out this study that found productivity increases during bad weather :)
That's all folks!
- I'm not even gonna attempt to summarize this, so here's the title: Multiple-Group confirmatory factor analysis in R – A tutorial in measurement invariance with continuous and ordinal indicators
- Improving exploratory factor analysis for ordinal data
- Improving multidimensional adaptive testing
Last but not least, it's not related to recruitment or assessment, but check out this study that found productivity increases during bad weather :)
That's all folks!
Monday, February 17, 2014
Research update
Okay, past time for another research update, so let's catch up!
Let's start with the Journal of Applied Social Psychology (Dec-Feb):
- Cultural intelligence plays a key role in multicultural teams
- Theory of Planned Behavior can be used to explain intent to submit video resumes
- More on weight-based discrimination, including additional evidence that this occurs more among women (free right now!)
- Does the physical attractiveness bias hold in same-sex evaluative situations? Not so much, although it may depend on someone's social comparison orientation
- "Dark side" traits play a role in predicting career preference
- Evidence that efficacy beliefs play a significant role not only in individual performance, but in team performance
Next up, the January issue of JAP:
- The concept of differential validity among ethnic groups in cognitive ability testing has been much debated, and this study adds to the discussion by suggesting that the effects are largely artifactual due to range restriction.
- Or are they? This study on the same topic found that range restriction could not account for observed differential validity findings. So the debate continues...
- A suggestion for how to increase the salience of dimension ratings in assessment centers
- Ambition and emotional stability appear related to adaptive performance, particularly for managers
Spring Personnel Psych (free right now!)
- First, a fascinating study of P-E fit across various cultures. Turns out relational fit may be more important in collectivistic and high power distance cultures (e.g., East Asia), whereas rational fit may be more important in individualistic and lower power distance cultures (e.g., the U.S.).
- Next, a study of recruitment messaging for job involving international travel.
- Last but definitely not least, a narrative and quantitative extensive review of the structured interview
Not quite done: One from Psychological Science on statistical power in testing mediation, and just in case you needed more evidence, the Nov/Dec issue of HRM has several research articles supporting the importance of line manager behavior and HRM practices on things like employee engagement.
The Spring issue of IJSA should be out soon, so see ya soon!
Let's start with the Journal of Applied Social Psychology (Dec-Feb):
- Cultural intelligence plays a key role in multicultural teams
- Theory of Planned Behavior can be used to explain intent to submit video resumes
- More on weight-based discrimination, including additional evidence that this occurs more among women (free right now!)
- Does the physical attractiveness bias hold in same-sex evaluative situations? Not so much, although it may depend on someone's social comparison orientation
- "Dark side" traits play a role in predicting career preference
- Evidence that efficacy beliefs play a significant role not only in individual performance, but in team performance
Next up, the January issue of JAP:
- The concept of differential validity among ethnic groups in cognitive ability testing has been much debated, and this study adds to the discussion by suggesting that the effects are largely artifactual due to range restriction.
- Or are they? This study on the same topic found that range restriction could not account for observed differential validity findings. So the debate continues...
- A suggestion for how to increase the salience of dimension ratings in assessment centers
- Ambition and emotional stability appear related to adaptive performance, particularly for managers
Spring Personnel Psych (free right now!)
- First, a fascinating study of P-E fit across various cultures. Turns out relational fit may be more important in collectivistic and high power distance cultures (e.g., East Asia), whereas rational fit may be more important in individualistic and lower power distance cultures (e.g., the U.S.).
- Next, a study of recruitment messaging for job involving international travel.
- Last but definitely not least, a narrative and quantitative extensive review of the structured interview
Not quite done: One from Psychological Science on statistical power in testing mediation, and just in case you needed more evidence, the Nov/Dec issue of HRM has several research articles supporting the importance of line manager behavior and HRM practices on things like employee engagement.
The Spring issue of IJSA should be out soon, so see ya soon!
Saturday, December 01, 2012
Research update
Cold/wet yet? Well sit back, have some hot tea, and let's catch up on our research...
Let's start with the biggie: the December issue of IJSA.
- Juggling selection quality and adverse impact continues to be tricky. The authors in this article suggest an optimum combination.
- Reflecting results from the turnover literature, this study found a relationship between perceptions of promotion practices and organizational justice and job satisfaction.
- Speaking of justice, the authors of this study found that the relationship between perceptions of distributive justice and intentions to recommend an employer were moderated by applicant affect.
- Job seekers would to well to keep up their psychological well being and self-esteem (easier said than done, right?).
- Back to fairness. It's reasonable to think that ethnic minority applicants may not perceive video resumes well (due to the increased salience of their minority status). But at least in this study, that assumption was not supported--although it depended on ethnic identity and language proficiency.
- Honestly, I'm not one for the pure concept of "multi-tasking": in my experience people perform in serial, not parallel. But that doesn't stop people from researching the predictive validity of multi-tasking assessments.
- Faking of personality inventories has been one of the hottest topics in assessment for years, but is often framed as dichotomies. This study looks to bring some needed complexity to the issue using qualitative data.
- We all know it can be challenging to get hiring manager to give up their unstructured interviews of questionable validity. Interestingly, according to this study, the more a hiring manager has used unstructured interviews, the less open they are to change. I don't know if this is a causality issue, a mediating variable issue, or more evidence of the inability to accurately judge one's abilities.
- Need more evidence for discrimination that occurs during resume screening? Here ya go.
- Understanding why certain individuals perform better during interviews is a needed area for study. In this article, the authors demonstrate the importance of simply being ability to generate ideas, rather than analyzing the situational requirements.
- This study investigates another area needing more attention: the selection into medical training and education programs. The authors found strong predictive support for SJTs but they came with lower face validity.
- Last in this issue is another interesting study, this time of 360-ratings of innovation. Interestingly and unlike a lot of other research on self-perception, self-ratings were lower than overall observer ratings. However, the situation got more complex when the authors separated and analyzed by level of self-rating.
Next the November issue of Journal of Applied Social Psychology:
- One of the most pervasive (yet bizarrely under-discussed) areas of discrimination in the workplace is age discrimination. This study illustrates some of the stereotypes held of younger and older workers--by both groups. (Spoiler alert: you'll find out how accurate some of these are in just a second)
- I've been waiting for this one, partly because I love hearing about how bizarre and non-face valid some puzzle-based interviews are. This particular study was looking at perceptions of these interviews compared to a behavioral interview. Results? The puzzle-based interviews were consistently less popular. Oh, did I mention that they didn't work as well?
How about the Winter issue of Personnel Psych?
- Why do certain applicants withdraw from the recruitment process? This study suggests a relationship with organizational identification.
- Okay, back to stereotypes about older workers. Whereas the earlier study looked at what the stereotypes are, this one looks at whether they're true. The answer: no, but for one: older workers are less willing to participate in training and career development.
- Turns out it's not just the unemployed that are frustrated by the job search process--currently employed individuals feel the same in many ways. Boy it's too bad we don't have a giant shared database that is able to match job demands with worker abilities...wait...
The November issue of Journal of Applied Psychology has a couple gems:
- Evidence for the predictive validity of the external manifestations of personality as well as the associated implicit motives.
- A reminder that what makes for effective leadership behavior depends on the culture.
The latest issue of Personnel Review has an interesting research article on utility analysis, where the authors reiterate how challenging it can be to communicate UA information (hint: carrot and stick approach may work best).
There are a couple good ones in the November issue of Psychological Science.
- Multiple-choice tests have been beat up in the past for being nothing more than tests of recognition (rather than productive retrieval). This study presents evidence that refutes that assumption. Go multiple-choice!
- Conspiracy theorists take note: governments may be less likely to use the assumed relationship between genetic testing and intelligence to pigeon-hole us into tracks. Why? Because, at least according to this study, there appears to be little evidence connecting the two.
Okay, this one is pretty cool--in a slightly scary way. The authors were looking at the impact that virtual avatar attractiveness has on interview ratings. Turns out our bias toward attractive people is so strong it extends to the virtual world! Of course maybe I should have seen that coming...I mean, ever read a comic book? (hat tip)
Still with me? Last but not least, some disturbing new evidence regarding significance testing and potential publication bias (hat tip). I'm guessing most of you won't be surprised at the finding.
I don't know if I'll have another update before the end of the year, so if I don't, happy holidays to everyone!
Saturday, May 26, 2012
May mega research update
It's time once again for the monthly research round-up. So let's dive right in:
The June International Journal of Selection and Assessment doesn't disappoint; let's take a look:
- More evidence of the link between personality variables and CWBs; this time with concurrent data in China
- Dovetailing nicely with a post I've been working on regarding promotional testing, this research indicates some interesting characteristics of internal test takers
- Why are supervisors open to behavioral interviews but shun discussion of "structure"? Looks like how we communicate about them plays a big role.
- More research on self-efficacy, this time teasing apart the concept a bit.
- Always a popular topic: applicant reactions to selection mechanisms. This time with a sample from Saudi Arabia.
- Speaking of applicant reactions...how about another study? This one comparing U.S. and Vietnamese college students. By the way, not surprisingly work samples came out a winner in both of these studies.
- Next, a fascinating study of a hidden bonus to UIT: despite the cheating element, it likely increases your candidate pool and eventually performance outcomes
- Speaking of response distortion, here's another study, this time of military cadet selection using personality inventories
- Okay, one more on inflation. This time a study of Chinese applicants--no difference compared to American samples.
- Back in March I wrote about a study Jeremy Bernerth published in J.A.P. that got a lot of attention. This time, Bernerth studied ethnic differences and found minority status was negatively related to credit scores.
Moving on to the summer issue of Personnel Psychology:
- The "file drawer problem" is the theory that nonsignificant results are less likely to get published. According to this study, that appears unlikely. But IMHO looking at all correlations is different than looking at the correlations key to one's hypothesis(es)...
- Back to faking (that may be this post's theme!), can response elaboration reduce faking on biodata items? This study suggests so. Although I'm left wondering...what was the impact on validity?
- Speaking of biodata, there are various ways of keying these items. This research suggests the best method depends on your sample size, although rational keying performed the worst.
How about the May issue of Journal of Applied Psychology?
- Well this is interesting...Chad Van Iddekinge and his colleagues have provided an updated meta-analysis on the criterion-related validity of integrity tests. What did they find? Well, the results appear to be less promising than those published previously (e.g., corrected r=.18 for job performance). Much like SIOP's research journal, this time J.A.P. published several commentaries in response to the study that...well, let's just say a debate ensued about the analysis...
- The Dark Triad. It sounds like something in a Dan Brown novel. But in this meta-analysis the authors show that personality characteristics that make up this triad (Machiavellianism, narcissism, and psychopathy) explain some variance in CWBs.
- Why are some people more proactive in seeking career goals than others? It's an important and under-researched question. In this study the authors show that part of the explanation lies in "future work selves", or how people's hopes and aspirations as they relate to work.
- Think self-reports of CWBs are biased? Perhaps not, according to this new study.
- Interested in what causes proactive customer service behavior? According to this multi-national study, self-efficacy is a key (along with service climate).
- Why do some leaders engage in more self-interested behavior than others? Perhaps not surprisingly, it appears due in part to the strength of their moral identity.
The May issue of the Journal of Applied Social Psychology has a couple gems...
- Hey, look, turns out being sensitive to your subordinates pays off. Talk about a lesson that needs frequent repeating...
- And that's it. Oh, wait, just this little study about using Facebook profiles to predict job performance...that I wrote about before....available in FULL right now...
Okay, getting to the end...The May/June issue of HRM:
- An interesting study of adverse impact in promotion decisions for managers in a Fortune 500 retailer. The authors compared three methods (top-down assessment, assessment centers, and multisource appraisal) and the results demonstrate how complex these analyses are!
- Speaking of complex. Think that successful job postings on the web is just fancy graphics? Think again--it still involves some classic factors like the labor market, firm reputation, and compensation incentives. The more things change...
- Identifying future leaders. There are few other issues that are as important for most organizations. Yet how exactly to do it eludes many. These authors propose a model that focuses on four main features: analytical ability, learning agility, drive, and emergent leadership.
Finally, a few from PARE:
- Does item order impact response anxiety? Not according to this study.
- What's that? How do we use a new jacknife procedure to eliminating items and improve structural equation modeling? You're in luck.
- Looks like a lot of research rely on beta weights when interpreting and reporting multiple linear regression results. But there's so much more...
Happy reading!
The June International Journal of Selection and Assessment doesn't disappoint; let's take a look:
- More evidence of the link between personality variables and CWBs; this time with concurrent data in China
- Dovetailing nicely with a post I've been working on regarding promotional testing, this research indicates some interesting characteristics of internal test takers
- Why are supervisors open to behavioral interviews but shun discussion of "structure"? Looks like how we communicate about them plays a big role.
- More research on self-efficacy, this time teasing apart the concept a bit.
- Always a popular topic: applicant reactions to selection mechanisms. This time with a sample from Saudi Arabia.
- Speaking of applicant reactions...how about another study? This one comparing U.S. and Vietnamese college students. By the way, not surprisingly work samples came out a winner in both of these studies.
- Next, a fascinating study of a hidden bonus to UIT: despite the cheating element, it likely increases your candidate pool and eventually performance outcomes
- Speaking of response distortion, here's another study, this time of military cadet selection using personality inventories
- Okay, one more on inflation. This time a study of Chinese applicants--no difference compared to American samples.
- Back in March I wrote about a study Jeremy Bernerth published in J.A.P. that got a lot of attention. This time, Bernerth studied ethnic differences and found minority status was negatively related to credit scores.
Moving on to the summer issue of Personnel Psychology:
- The "file drawer problem" is the theory that nonsignificant results are less likely to get published. According to this study, that appears unlikely. But IMHO looking at all correlations is different than looking at the correlations key to one's hypothesis(es)...
- Back to faking (that may be this post's theme!), can response elaboration reduce faking on biodata items? This study suggests so. Although I'm left wondering...what was the impact on validity?
- Speaking of biodata, there are various ways of keying these items. This research suggests the best method depends on your sample size, although rational keying performed the worst.
How about the May issue of Journal of Applied Psychology?
- Well this is interesting...Chad Van Iddekinge and his colleagues have provided an updated meta-analysis on the criterion-related validity of integrity tests. What did they find? Well, the results appear to be less promising than those published previously (e.g., corrected r=.18 for job performance). Much like SIOP's research journal, this time J.A.P. published several commentaries in response to the study that...well, let's just say a debate ensued about the analysis...
- The Dark Triad. It sounds like something in a Dan Brown novel. But in this meta-analysis the authors show that personality characteristics that make up this triad (Machiavellianism, narcissism, and psychopathy) explain some variance in CWBs.
- Why are some people more proactive in seeking career goals than others? It's an important and under-researched question. In this study the authors show that part of the explanation lies in "future work selves", or how people's hopes and aspirations as they relate to work.
- Think self-reports of CWBs are biased? Perhaps not, according to this new study.
- Interested in what causes proactive customer service behavior? According to this multi-national study, self-efficacy is a key (along with service climate).
- Why do some leaders engage in more self-interested behavior than others? Perhaps not surprisingly, it appears due in part to the strength of their moral identity.
The May issue of the Journal of Applied Social Psychology has a couple gems...
- Hey, look, turns out being sensitive to your subordinates pays off. Talk about a lesson that needs frequent repeating...
- And that's it. Oh, wait, just this little study about using Facebook profiles to predict job performance...that I wrote about before....available in FULL right now...
Okay, getting to the end...The May/June issue of HRM:
- An interesting study of adverse impact in promotion decisions for managers in a Fortune 500 retailer. The authors compared three methods (top-down assessment, assessment centers, and multisource appraisal) and the results demonstrate how complex these analyses are!
- Speaking of complex. Think that successful job postings on the web is just fancy graphics? Think again--it still involves some classic factors like the labor market, firm reputation, and compensation incentives. The more things change...
- Identifying future leaders. There are few other issues that are as important for most organizations. Yet how exactly to do it eludes many. These authors propose a model that focuses on four main features: analytical ability, learning agility, drive, and emergent leadership.
Finally, a few from PARE:
- Does item order impact response anxiety? Not according to this study.
- What's that? How do we use a new jacknife procedure to eliminating items and improve structural equation modeling? You're in luck.
- Looks like a lot of research rely on beta weights when interpreting and reporting multiple linear regression results. But there's so much more...
Happy reading!
Sunday, November 20, 2011
Research update #583: Impression management and a lot more
Okay, I've got a lot of ground to cover this time, so buckle up...
Let's start with the December issue of IJSA:
- Looks like how much applicants try to make themselves look good varies by country
- Is applicant faking behavior related to job performance? Kinda depends on your definitions.
- Research has found that emotional intelligence can be related to work attitudes. This appears to be in part because of an increased situational judgment effectiveness.
- Speaking of situational judgment...in terms of job knowledge, knowing what to do is different than knowing what not to do
- What impact does a resume have on a recruiter? Depends on what assumptions they make about you after reading it.
- How to people select--and continue with--an executive coach? By looking at things like their ability to forge a partnership.
- How do Canadian firms do in terms of using tests other than interviews? Not so well, it turns out.
Let's move to the October issue of JASP, where there's just one article but it's a good one. Researchers continued the (depressing) finding that applicant names impact pre-interview impressions. Specifically, the more a name was Anglicized, the more favorable the impression was when hiring for an outside sales job.
Next comes the November issue of JAP:
- A new meta-analysis of the FFM of personality and its relationship to OCBs and task performance.
- Measures of interest haven't gotten a lot of love as selection devices. Looks like we need to tease out the constructs a little because they could be more helpful than we thought.
- Applicants trying to create a certain image during an interview are better off doing this after an initial flub or relying solely on self-promotion rather than making up an image.
A few from the November issue of JPSP:
- Another on impression management (not selection-specific) that goes into more detail about the topic (e.g., how many tactics people use, their accuracy)
- A caution about using the Revised NEO-PI in different cultures due to DIF.
Next, a call for more transparency in false-positive findings.
Last but not least, those of you interested in the potential of social ratings of performance being used for selection might be interested in this study of RateMyProfessors.com, which found student ratings are likely to be useful measures of teacher quality.
Let's start with the December issue of IJSA:
- Looks like how much applicants try to make themselves look good varies by country
- Is applicant faking behavior related to job performance? Kinda depends on your definitions.
- Research has found that emotional intelligence can be related to work attitudes. This appears to be in part because of an increased situational judgment effectiveness.
- Speaking of situational judgment...in terms of job knowledge, knowing what to do is different than knowing what not to do
- What impact does a resume have on a recruiter? Depends on what assumptions they make about you after reading it.
- How to people select--and continue with--an executive coach? By looking at things like their ability to forge a partnership.
- How do Canadian firms do in terms of using tests other than interviews? Not so well, it turns out.
Let's move to the October issue of JASP, where there's just one article but it's a good one. Researchers continued the (depressing) finding that applicant names impact pre-interview impressions. Specifically, the more a name was Anglicized, the more favorable the impression was when hiring for an outside sales job.
Next comes the November issue of JAP:
- A new meta-analysis of the FFM of personality and its relationship to OCBs and task performance.
- Measures of interest haven't gotten a lot of love as selection devices. Looks like we need to tease out the constructs a little because they could be more helpful than we thought.
- Applicants trying to create a certain image during an interview are better off doing this after an initial flub or relying solely on self-promotion rather than making up an image.
A few from the November issue of JPSP:
- Another on impression management (not selection-specific) that goes into more detail about the topic (e.g., how many tactics people use, their accuracy)
- A caution about using the Revised NEO-PI in different cultures due to DIF.
Next, a call for more transparency in false-positive findings.
Last but not least, those of you interested in the potential of social ratings of performance being used for selection might be interested in this study of RateMyProfessors.com, which found student ratings are likely to be useful measures of teacher quality.
Thursday, April 07, 2011
A little of this, a little of that
I've got a hodgepodge of things for you this time, some research mixed with some other interesting things.
First, the March issue of Journal of Applied Psychology, which has been out for a while and I've been a little slow to get to:
- Becker and Cropanzano on the (non-linear) relationship between job performance and voluntary turnover (people tend to skedaddle when they're on a performance skid).
- Podsakoff, et al. with a fascinating study of the impact of OCBs on interviewee ratings. Turns out they made quite an impact, particularly for the higher-level position, and particularly when candidates demonstrated low levels of OCB (e.g., helping, loyalty). Raters were probably surprised that a situation that so clearly calls for impression management failed to elicit it.
- McDaniel, et al. with a great piece on SJTs. The authors describe two adjustments that can be made to traditional SJTs that improve validity, reduce Black-White mean differences and score elevation due to coaching, and reduce total length. That all sounds pretty good to me, and you can read an in press version here.
- Last but not least, Swider, et al. report the results of a study on job search effort and voluntary turnover. Job embededness appears to play an important rule, as does job satisfaction and the availability of alternatives.
Speaking of McDaniel, he and his colleagues have written an article for an upcoming issue of Industrial and Organizational Psychology with the provocative title of "The Uniform Guidelines are a detriment to the field of personnel selection." SIOP members should be sure to consider submitting a commentary, and even if you're not a member, you should check out the in press version; it's a good read.
Speaking of adverse impact...I attended a fascinating webinar sponsored by PTC/MC on Wednesday where Kenneth Yusko of Marymount University described the development of the Siena Reasoning Test, which uses a slightly different question type (along with some other techniques) to reduce d but maintain criterion-related validity. Provocative stuff, and one of the the holy grails of personnel assessment. Which probably explains why Yusko and his colleagues are being presented with the 2011 M. Scott Myers award at this year's SIOP conference. Interested? Check it out yourself. You can also flip through slides from a similar presentation at the 2008 IPAC conference.
Speaking of IPAC, have you registered for the July conference in D.C.? It's shaping up to be another great one--just check out some of the pre-conference workshops.
Finally, if you're up for a little heated discussion, head on over to ERE, where Wendell Williams laments about the increasing number of people who claim to be "experts" in assessment but who lack the chops. He particularly calls out poorly informed bloggers. Hey...wait a minute...
First, the March issue of Journal of Applied Psychology, which has been out for a while and I've been a little slow to get to:
- Becker and Cropanzano on the (non-linear) relationship between job performance and voluntary turnover (people tend to skedaddle when they're on a performance skid).
- Podsakoff, et al. with a fascinating study of the impact of OCBs on interviewee ratings. Turns out they made quite an impact, particularly for the higher-level position, and particularly when candidates demonstrated low levels of OCB (e.g., helping, loyalty). Raters were probably surprised that a situation that so clearly calls for impression management failed to elicit it.
- McDaniel, et al. with a great piece on SJTs. The authors describe two adjustments that can be made to traditional SJTs that improve validity, reduce Black-White mean differences and score elevation due to coaching, and reduce total length. That all sounds pretty good to me, and you can read an in press version here.
- Last but not least, Swider, et al. report the results of a study on job search effort and voluntary turnover. Job embededness appears to play an important rule, as does job satisfaction and the availability of alternatives.
Speaking of McDaniel, he and his colleagues have written an article for an upcoming issue of Industrial and Organizational Psychology with the provocative title of "The Uniform Guidelines are a detriment to the field of personnel selection." SIOP members should be sure to consider submitting a commentary, and even if you're not a member, you should check out the in press version; it's a good read.
Speaking of adverse impact...I attended a fascinating webinar sponsored by PTC/MC on Wednesday where Kenneth Yusko of Marymount University described the development of the Siena Reasoning Test, which uses a slightly different question type (along with some other techniques) to reduce d but maintain criterion-related validity. Provocative stuff, and one of the the holy grails of personnel assessment. Which probably explains why Yusko and his colleagues are being presented with the 2011 M. Scott Myers award at this year's SIOP conference. Interested? Check it out yourself. You can also flip through slides from a similar presentation at the 2008 IPAC conference.
Speaking of IPAC, have you registered for the July conference in D.C.? It's shaping up to be another great one--just check out some of the pre-conference workshops.
Finally, if you're up for a little heated discussion, head on over to ERE, where Wendell Williams laments about the increasing number of people who claim to be "experts" in assessment but who lack the chops. He particularly calls out poorly informed bloggers. Hey...wait a minute...
Sunday, March 13, 2011
Research roundup: Political skill, emotions, and...hobos?
Time to catch up on the research:
First, Blickle, et al. show that political skill ("the capacity to understand others in working life effectively, and to apply such knowledge to induce others to act in ways that add to one's personal or organizational goals") can predict job performance beyond GMA and personality factors.
Fast on its heels comes a study from Bing et al. that also shows support for the concept of political skill (what is it, election season?). This time the researchers use meta-analysis to show a positive relationship between political skill and both task and contextual performance, with a stronger link to the latter. The relationship is also stronger as interpersonal and social requirements of the job increase.
Next, Giordano, et al. provide evidence that can help organizations detect deception in interviews (hint: reviewers out-performed interviewers, and giving a warning helps).
Fukuda, et al. show that the factor structure of two emotional intelligence measures remained consistent when translated into Japanese.
Hjemdal et al. continue gathering support for a measure of resilience (essentially the ability to successfully deal with stressors); this time in a French-speaking Belgian sample.
Interested in the Bookmark method of setting cut scores? Then you'll want to take a look at this study by Davis-Becker et al. and its slightly disturbing results.
The measurement of reading comprehension is common, and it's important to understand the construct(s) being measured. This piece by Svetina, et al. helps us parse out the concept.
The discussion about the March issue of the Journal of Personality and Social Psychology was dominated by Bem's study of precognition, but a hidden gem is there to be found: Hopwood et al.'s study of how personality changes over one's lifespan. Looks like it's largely due to environmental influences and the most change occurs early on.
Interested in PO fit and its relationship with attitudinal outcomes? Then you'll want to check out Leung & Chaturvedi's study.
Last but not least, Woo analyzes a sample of U.S. workers and finds support for "Ghiselli's hobo syndrome" among a small group: basically these folks frequently change jobs and have positive attitudes about it. Not surprisingly, these hobos reported being less satisfied with their current jobs. I guess there's just no pleasing some people!
Side note: I just ran across the Questionmark blog. A lot of really cool stuff that focuses on education and learning applications, but a lot of the content is directly applicable to personnel assessment. They're a vendor, so there are some sales pitches, but it's worth weeding through.
Final note: I apologize ahead of time if some of these links don't work; I've tried my best to link to non-session based abstracts but it can be hard to tell.
Sunday, February 27, 2011
Spring '11 P-Psych: Leadership, interviews, competency modeling
It's journal season--this time let's take a look at the Spring 2011 issue of Personnel Psychology:
First up, Derue, et al. with an important meta-analysis on leadership effectiveness. After looking at 59 studies, they found that leader traits and behaviors explained a minimum of 31% of leadership effectiveness. Interestingly, group performance (which I would argue is the most important criterion) was the most difficult to predict. Behaviors tended to explain more than traits, but the authors suggest a model where behavior mediates the relationship between traits and effectiveness is warranted. Not surprisingly, the best trait predictor depended on the criterion: conscientiousness predicted leader effectiveness, group performance, and follower job satisfaction the best, while satisfaction with leader was best predicted by leader agreeableness (reminds me of a recent IJSA study). The same was true with leader behaviors, although consideration was a good predictor across the criteria. A must for anyone interested in leadership research, and you can read an in-press version here.
Next, a piece by Melchers, et al. on whether more interview structure really leads to better rating quality (I'll ruin it for you: yes). Specifically, using a sample of primarily undergraduates, the authors found that providing subjects with frame-of-reference (FOR) training and descriptively anchored rating scales led to substantial increases in rater accuracy and interrater reliability. You can read an in-press version here.
Those of you interested in the concept of core self-evaluations will want to read the study by Ferris, et al.
One area that we don't see enough research in is newcomer adaptation. The longitudinal study by Wang, et al. of a group of Chinese subjects helps fill that hole by exploring the relationship between adaptability, person-environment fit, and work-related outcomes.
Last but most certainly not least, Campion, et al. provide us with a review of best practices in competency modeling. Specifically, 20 of them. Of particular note to you may be that they distinguish competency modeling from job analysis. Want to read the whole thing? Me too! Wish I could have found an in-press version but no such luck.
Sidenote: while not specifically related to recruitment or selection, Christian, et al.'s piece on engagement may be of interest to several readers.
First up, Derue, et al. with an important meta-analysis on leadership effectiveness. After looking at 59 studies, they found that leader traits and behaviors explained a minimum of 31% of leadership effectiveness. Interestingly, group performance (which I would argue is the most important criterion) was the most difficult to predict. Behaviors tended to explain more than traits, but the authors suggest a model where behavior mediates the relationship between traits and effectiveness is warranted. Not surprisingly, the best trait predictor depended on the criterion: conscientiousness predicted leader effectiveness, group performance, and follower job satisfaction the best, while satisfaction with leader was best predicted by leader agreeableness (reminds me of a recent IJSA study). The same was true with leader behaviors, although consideration was a good predictor across the criteria. A must for anyone interested in leadership research, and you can read an in-press version here.
Next, a piece by Melchers, et al. on whether more interview structure really leads to better rating quality (I'll ruin it for you: yes). Specifically, using a sample of primarily undergraduates, the authors found that providing subjects with frame-of-reference (FOR) training and descriptively anchored rating scales led to substantial increases in rater accuracy and interrater reliability. You can read an in-press version here.
Those of you interested in the concept of core self-evaluations will want to read the study by Ferris, et al.
One area that we don't see enough research in is newcomer adaptation. The longitudinal study by Wang, et al. of a group of Chinese subjects helps fill that hole by exploring the relationship between adaptability, person-environment fit, and work-related outcomes.
Last but most certainly not least, Campion, et al. provide us with a review of best practices in competency modeling. Specifically, 20 of them. Of particular note to you may be that they distinguish competency modeling from job analysis. Want to read the whole thing? Me too! Wish I could have found an in-press version but no such luck.
Sidenote: while not specifically related to recruitment or selection, Christian, et al.'s piece on engagement may be of interest to several readers.
Thursday, February 17, 2011
March IJSA: So much good stuff
The March 2011 issue of the International Journal of Selection and Assessment is out, and it's a doozie. Check it out:- Beaty, et al. present evidence that unproctored Internet tests (noncognitive: personality and biodata) generally had similar criterion-related validities across a spectrum of job performance criteria compared to the same administered in a proctored setting. Mmmm....UIT...
- Generally when people re-take a cognitive ability test, they do better. But do they do better on other tests of cognitive ability? Matton, et al. describe a study that looked at just that and found the answer to be "no."
- One advantage that is frequently claimed about personality inventories is that usage results in less adverse impact compared to, say, ability or knowledge tests. But might the AI depend on how hires are made (e.g., top-down, compensatory)? Turns out the answer is yes--at least according to some results by Risavy and Hausdorf.
- Hey, government agencies, still debating whether to put more resources into your career web portal? Maybe this will convince you. Selden and Orenstein show that governments with more usable portals as well as better available content not only attract more applicants per opening, but have less voluntary turnover of new hires.
- With advances in technology and changes to the work environment, clerical jobs have changed a lot over the last 30 years and the old ways of selecting for these jobs (namely g-loaded tests such as perceptual speed and verbal ability) likely need to be re-thought...right? Well, not so much, at least according to a meta-analysis by Whetzel and her colleagues. In fact, the criterion-related validity values met or exceeded those found 30+ years ago. The more things change...
- De Goede, et al. present the results of a study that explores the relationship between P-O fit and organizational websites but include the concept of person-industry fit. One implication: if you're trying to attract a more diverse group of candidates, work on making your portal more attractive.
- We spend a lot of time trying to make sure interviews are loaded with job-relevant content. But how much attention do we pay to the impact of impression management tactics on the part of applicants? Huffcutt's results make a compelling argument that we ignore the latter to our detriment as it may have more to do with interview ratings than the job-relevant content.
- How does one determine managerial potential? Well, it depends who you ask. Thomason, et al. present results that indicate when supervisors are asked, they focus on task-based personality traits (e.g., conscientiousness), whereas peers focus on contextual traits such as agreeableness. Given that leadership is ultimately about achieving things through subordinates, I wonder what we should be paying attention to....hmmm...how about both?
- Thinking about using self-ratings of political skill as part of the application process? I can certainly see situations where this skill may be helpful, but might this method be susceptible to inflation? Not so much, at least according to results from Blickle, et al.
- Last but definitely not least, Carless & Hetherington with some data on the impact of recruitment timeliness on applicant attraction. The longer we make applicants wait, the less attracted to the organization they will be, right? Not so fast. According to this research, it is perceived timeliness that matters, not actual timeliness (hence the importance of communication). In addition, this relationship is partially mediated by job and organizational characteristics.
Tuesday, November 02, 2010
November '10 J.A.P.
The November 2010 issue of the Journal of Applied Psychology is out, so let's take a look at the relevant articles:
Woods & Hampson report the results of a fascinating study looking at the relationship between childhood personality and adult occupational choice. The participants (N ~ 600) were given a 5-factor personality inventory when they were between 6 and 12 years old, then reported their occupation 40 years later using Holland's vocational types. Results? Openness/Intellect and Conscientiousness scores were correlated with occupational choice as adults. Implication? Career choice may be determined fairly early on in some cases and be influenced by how we're hard-wired as well as through friends, family, etc.
Perhaps even more interesting, the authors found that for the most strongly sex-typed work environments (think construction, nursing), the results related to Openness/Intellect were moderated by gender, supporting the idea that gender stereotyping in these jobs is impacted by individual differences as well as other factors (e.g., cultural norms).
Next up, a study by Maltarich, et al. on the relationship between cognitive ability and voluntary turnover, with some (what I thought were) counter-intuitive results. The authors focused on cognitive demands as the moderating factor, so think for a second about what you would expect to find for a job with high cognitive demands (think lawyer)--who would be most likely to leave, those with high cognitive ability or those with low? I naturally assumed the latter.
Turns out it was neither. The authors found a curvilinear relationship, such that those low and high in cognitive ability were more likely to leave than those in the middle.
What about jobs lower in cognitive demands? Who would you expect to have higher voluntary turnover--those with high cognitive ability or lower cognitive ability? I assumed high, and again I was wrong. Turns out the relationship the authors found in that case was more of a straight negative linear relationship: the higher the cognitive demands, the less likely to leave.
What might explain these relationships? Check out I/O at Work's post about this article for more details including potential explanations from the authors. It certainly has implications for selection decisions based on cognitive ability scores (and reminds me of Jordan v. New London).
Do initial impressions of candidates matter during an interview? The next article, by Barrick et al. helps us tease out the answer. The authors found that candidates that made a better impression in the opening minutes of an interview received higher interview scores (r=.44) and were more likely to receive an internship offer (r=.22). Evaluations of initial competence impacted interview outcomes not only with the same interviewer, but with separate interviewers, and even separate interviewers who skipped rapport building.
But perhaps more interestingly, the authors found that assessments of candidate liking and similarity were not significantly related to other judgments made by separate interviewers. Thus, while these results support the idea that initial impression matter, they also provide strong support for using a panel interview with a diverse makeup, so that bias unrelated to competence is less likely to influence selection decisions.
Finally, Dierdorff et al. describe the results of a field study that looked at who might benefit most from frame-of-reference training (FOR). As a reminder, FOR is used to provide raters with a context for their rating and typically involves discussing the multi-dimensional nature of performance and rating anchors, and conducting practice sessions with feedback to increase accuracy and effectiveness (Landy & Conte, 2009).
In this case, the authors were interested in finding out whether individual differences might impact the effectiveness of FOR. What they found was that the negative impact of having a motivational tendency to avoid performance can be mitigated by having higher levels of learning self-efficacy. In other words, FOR training may be particularly effective for individuals that believe they are capable of learning and applying training, and overall results may be enhanced by encouraging this belief among all the raters.
References:
Landy, F. and Conte, J. (2009). Work in the 21st century: An introduction to industrial and organizational psychology (Third Edition). Hoboken, NJ: Wiley-Blackwell.
Woods & Hampson report the results of a fascinating study looking at the relationship between childhood personality and adult occupational choice. The participants (N ~ 600) were given a 5-factor personality inventory when they were between 6 and 12 years old, then reported their occupation 40 years later using Holland's vocational types. Results? Openness/Intellect and Conscientiousness scores were correlated with occupational choice as adults. Implication? Career choice may be determined fairly early on in some cases and be influenced by how we're hard-wired as well as through friends, family, etc.
Perhaps even more interesting, the authors found that for the most strongly sex-typed work environments (think construction, nursing), the results related to Openness/Intellect were moderated by gender, supporting the idea that gender stereotyping in these jobs is impacted by individual differences as well as other factors (e.g., cultural norms).
Next up, a study by Maltarich, et al. on the relationship between cognitive ability and voluntary turnover, with some (what I thought were) counter-intuitive results. The authors focused on cognitive demands as the moderating factor, so think for a second about what you would expect to find for a job with high cognitive demands (think lawyer)--who would be most likely to leave, those with high cognitive ability or those with low? I naturally assumed the latter.
Turns out it was neither. The authors found a curvilinear relationship, such that those low and high in cognitive ability were more likely to leave than those in the middle.
What about jobs lower in cognitive demands? Who would you expect to have higher voluntary turnover--those with high cognitive ability or lower cognitive ability? I assumed high, and again I was wrong. Turns out the relationship the authors found in that case was more of a straight negative linear relationship: the higher the cognitive demands, the less likely to leave.
What might explain these relationships? Check out I/O at Work's post about this article for more details including potential explanations from the authors. It certainly has implications for selection decisions based on cognitive ability scores (and reminds me of Jordan v. New London).
Do initial impressions of candidates matter during an interview? The next article, by Barrick et al. helps us tease out the answer. The authors found that candidates that made a better impression in the opening minutes of an interview received higher interview scores (r=.44) and were more likely to receive an internship offer (r=.22). Evaluations of initial competence impacted interview outcomes not only with the same interviewer, but with separate interviewers, and even separate interviewers who skipped rapport building.
But perhaps more interestingly, the authors found that assessments of candidate liking and similarity were not significantly related to other judgments made by separate interviewers. Thus, while these results support the idea that initial impression matter, they also provide strong support for using a panel interview with a diverse makeup, so that bias unrelated to competence is less likely to influence selection decisions.
Finally, Dierdorff et al. describe the results of a field study that looked at who might benefit most from frame-of-reference training (FOR). As a reminder, FOR is used to provide raters with a context for their rating and typically involves discussing the multi-dimensional nature of performance and rating anchors, and conducting practice sessions with feedback to increase accuracy and effectiveness (Landy & Conte, 2009).
In this case, the authors were interested in finding out whether individual differences might impact the effectiveness of FOR. What they found was that the negative impact of having a motivational tendency to avoid performance can be mitigated by having higher levels of learning self-efficacy. In other words, FOR training may be particularly effective for individuals that believe they are capable of learning and applying training, and overall results may be enhanced by encouraging this belief among all the raters.
References:
Landy, F. and Conte, J. (2009). Work in the 21st century: An introduction to industrial and organizational psychology (Third Edition). Hoboken, NJ: Wiley-Blackwell.
Monday, October 25, 2010
Don't ask candidates to judge themselves
Imagine you're buying a car. The salesperson throws out a price on the car you're interested in. And here are the questions you ask to try to determine whether it's a good deal:
- Is this a good price?
- How good of a salesperson are you?
- Compared to other sales you've made, how good is this one?
Think this is silly? Well it's essentially what many employers are doing when they interview people or otherwise rely on descriptions of experience when screening. They rely way too heavily on self-descriptions when they should be taking a more rigorous approach. Think these types of questions, which should be stricken from your inventory:
"What's something you're particularly good at?"
"How would you describe your skills compared to other people?"
There are two main problems with asking these types of questions in a high-stakes situation like a job interview (or buying a car):
1) People are motivated to inflate their answers, or just plain lie, in these situations. You know that. I know that. But it's surprising how many people forget it.
2) People are bad at accurately describing themselves. We know this from years of research, but if you're interested, check out a recent study published in the Journal of Personality and Social Psychology that compared Big 5 personality ratings from four countries and found that people generally hold more favorable opinions of themselves compared to how others see them.
But it's even worse than it appears. It's not just that people inflate themselves, it's that some of your best candidates deflate themselves. Think about your star performers: if you asked them how good they were in a particular area, what do you think they'd say?
You essentially want to know two things about candidates:
1) What they've done
2) What they're capable of doing
To answer the first issue, you have several options, including:
1a) Asking them to describe what they've done--the so-called "behavioral interviewing" technique. Research shows that these types of questions generally contribute a significant amount of validity to the process. But they're not perfect by any means, particularly with people with bad memories about themselves. And keep in mind at that point you're taking their word for it.
1b) Asking them for examples of what they've done. Best used as a follow-up to a claim, but tricky in any situation where there's even a remote possibility that someone else did it or did most of it (so practically everything outside of the person being videotaped).
1c) Asking others (e.g., co-workers, supervisors) what the candidate's done. Probably the most promising but most difficult data to accurately capture. Hypothetically if the person has any job history at all they've left a trail of accomplishments and failures, as well as a reliable pattern of responding to situations. This is the promise of reference checks that so often is either squandered ("I don't have time") or stymied ("They just gave me name, title, and employment dates"). Don't use these excuses, investigate.
As for the second issue, you have several options as well, including:
2a) Asking knowledge-based questions in an interview. For whatever reason these seem to have fallen out of favor, but if there is a body of knowledge that is critical to know prior to employment, ask about it. At the worst you'll weed out those who have absolutely no idea.
2b) Using another type of assessment, such as a performance test/work sample, on-site writing exercise, role play, simulation, or written multiple choice test (to name a few). Properly developed and administered, these will give you a great sense of what people are capable of--just make sure the tests are tied back to true job requirements.
2c) Using the initial period of employment (some call it probation) to throw things at the person and see what they're capable of. It's important not test their ability to deal with overload (unless that's critical to the job), but get them involved in a diverse set of projects. Ask for their input. Ask them to do some research. See what they are capable of delivering, even if it's a little rough.
Whatever you do, triangulate on candidate knowledge, skills, and abilities. Use multiple measures to get an accurate picture of what they bring. Consider using an interview for a two-way job preview as much as an assessment device.
But above all, don't take one person's word for things. Unless you like being sold a lemon.
Friday, September 24, 2010
September mega research update
I'm behind in posting updates of the September journals. Really behind. So instead of posting a series describing the detailed contents of each issue, I'm going to give you links to what's come out in the last month or so and let you explore. But I'll try to hit some high points:
Journal of Applied Psychology, v95(5)
Highlights include this piece from Ramesh and Gelfand who studied call center employees in the U.S. and India and found that while person-job fit predicted turnover in the U.S., person-organization fit predicted turnover in India.
Human Performance, v23(4)
This issue has some good stuff, including Converse et al.'s piece on different forced-choice formats for reducing faking on personality measures, Perry et al.'s piece on better predicting task performance using the achievement facet of conscientiousness, and Zimmerman et al.'s article on observer ratings of performance.
Journal of Business and Psychology, v25(3)
Another issue filled with lots of good stuff, but I'm almost 100% positive abstract views are session based, so use the title link for article on response rates in organizational research (full text here), the importance of using multiple recruiting activities, and the importance of communicating benefit information in job ads.
Journal of Applied Social Psychology, v40(9)
The article to check out here is by Proost et al. and deals with different self-promotion techniques during an interview and their effect on interviewer judgments.
Journal of Occupational and Organizational Psychology, v83(3)
Check this issue out for articles on organizational attraction, communication apprehension in assessment centers, and the impact of interviewer affectivity on ratings.
Sunday, May 23, 2010
June 2010 IJSA
The summer journal season continues with the June 2010 issue of the International Journal of Selection and Assessment. Take a deep breath, there's a lot of stuff packed into this issue:
- Roth et al. provide evidence that women outperformed men on work sample exams that involved social skills, writing skills, or a broad array of KSAs. To the extent that an employer is trying to avoid discriminating against female applicants, this provides support for work sample usage.
- In a study of managers in Taiwan, Tsai et al. show that the most effective way an applicant can make up for a slip in an interview is to apologize (vs. attempting to justify or use an excuse).
- Jackson et al. strive to add some clarity on task-based assessment centers
- Blickle & Schnitzler provide evidence of the construct and criterion-related validity of the political skill inventory
- Colarelli et al. studied how racial prototypicality and affirmative action policies impact hiring decisions. Results of a resume review indicated more jobs were awarded to black candidates as racial prototypicality and affirmative action policy strength increased, but stronger AA policies decreased the percentage of minority hires attributed to higher qualifications.
- In my personal favorite article of the issue, Karl et al. found in a study of U.S. and German students that those low on conscientiousness (especially), agreeableness, and emotional stability were more likely to post "Facebook Faux Pas". This provides some support for employers who screen out applicants based on inappropriate social networking posts. I'll talk more about this in my upcoming webinar.
- Denis, et al. provide support for the NEO PI-R's ability to predict job performance in two French-Canadian samples.
- Bilgiç and Acarlar report results of a study of Turkish students and perceptions of various selection instruments. Interviews were rated most highly and there were some differences in terms of privacy perceptions depending on the goal orientation of the student.
- Trying to figure out how to hire better direct support professionals (e.g., those providing long-term residential care or care to those with disabilities)? Robson, et al. describe the development of a composite predictor composed of various measures (e.g., agreeableness, numerical ability) that predicted performance, satisfaction, and turnover.
- Ahmetoglu et al. provide support for using the Fundamental Interpersonal Relationship Orientations-Behaviour (FIRO-B) to predict leadership capability.
- Ispas et al. describe results of a study that showed support for a nonverbal cognitive ability measure (the GAMA) in predicting job performance in two samples.
- Last but not least, in another win for context-specific assessments, Pace & Brannick show how a measure of openness to experience tailored to specific work outpredicted the comparable general NEO PI-R scale. IMHO this is how personality measures will eventually become more prominent and accepted as pre-hire assessments.
- Roth et al. provide evidence that women outperformed men on work sample exams that involved social skills, writing skills, or a broad array of KSAs. To the extent that an employer is trying to avoid discriminating against female applicants, this provides support for work sample usage.
- In a study of managers in Taiwan, Tsai et al. show that the most effective way an applicant can make up for a slip in an interview is to apologize (vs. attempting to justify or use an excuse).
- Jackson et al. strive to add some clarity on task-based assessment centers
- Blickle & Schnitzler provide evidence of the construct and criterion-related validity of the political skill inventory
- Colarelli et al. studied how racial prototypicality and affirmative action policies impact hiring decisions. Results of a resume review indicated more jobs were awarded to black candidates as racial prototypicality and affirmative action policy strength increased, but stronger AA policies decreased the percentage of minority hires attributed to higher qualifications.
- In my personal favorite article of the issue, Karl et al. found in a study of U.S. and German students that those low on conscientiousness (especially), agreeableness, and emotional stability were more likely to post "Facebook Faux Pas". This provides some support for employers who screen out applicants based on inappropriate social networking posts. I'll talk more about this in my upcoming webinar.
- Denis, et al. provide support for the NEO PI-R's ability to predict job performance in two French-Canadian samples.
- Bilgiç and Acarlar report results of a study of Turkish students and perceptions of various selection instruments. Interviews were rated most highly and there were some differences in terms of privacy perceptions depending on the goal orientation of the student.
- Trying to figure out how to hire better direct support professionals (e.g., those providing long-term residential care or care to those with disabilities)? Robson, et al. describe the development of a composite predictor composed of various measures (e.g., agreeableness, numerical ability) that predicted performance, satisfaction, and turnover.
- Ahmetoglu et al. provide support for using the Fundamental Interpersonal Relationship Orientations-Behaviour (FIRO-B) to predict leadership capability.
- Ispas et al. describe results of a study that showed support for a nonverbal cognitive ability measure (the GAMA) in predicting job performance in two samples.
- Last but not least, in another win for context-specific assessments, Pace & Brannick show how a measure of openness to experience tailored to specific work outpredicted the comparable general NEO PI-R scale. IMHO this is how personality measures will eventually become more prominent and accepted as pre-hire assessments.
Friday, May 14, 2010
Personnel Psychology, Summer 2010
The latest issue of Personnel Psychology (v63, #2) marks the beginning of summer journal season. Let's take a peek at some of what's inside:
Practice makes...better. John Hausknecht studied over 15,000 candidates who applied for supervisory positions (and 357 who repeated the process) over a 4-year period with a large organization in the service industry. The selection process included a personality test. He found that candidates that failed the first time around showed practice effects on dimension-level scores of .40 to .60. Candidates that passed the first time, but were taking the test again for other reasons, generally showed no difference in scores. More interestingly, on several subscales low scores the first time around were associated with practice effects that exceeded one standard deviation. A good reminder that personality inventories are susceptible to "faking", but certainly not a nail in their coffin as they still work quite well in many situations.
Another reason to structure your interviews. As if you needed more convincing, McCarthy et al.'s study of nearly 20,000 applicants for a managerial-level position in a large organization found that the use of a structured interview resulted in zero main effects for applicant gender and race on interview performance. Similarly, there were no effects of applicant-interviewer similarity with respect to gender and race.
Users of the CRT-A take note. The conditional reasoning test of aggression (CRT-A) is used to detect individuals with a propensity for aggression. Previous studies have suggested the criterion-related validity of this test is around r=.44. In this study, by Berry et al., the authors meta-analyzed a large data set and found much lower values, in the .10-.16 range, that rose to .24-.26 when certain studies were excluded.
Assess your way into a job. Last but not least, Wanberg et al. describe the development of an inventory for job seekers called Getting Ready for your Next Job (YNJ, available here). The authors present results tying inventory components (e.g., job search intensity, Internet use) to subsequent employment outcomes.
Stay tuned, new issues of JAP, IJSA, and others should be out soon!
Saturday, February 20, 2010
Recruitment v. Assessment, Round 1: Fight!
I'm sure some of you are avid readers of ERE (Electronic Recruiters Exchange), but for those of you that aren't (and don't receive my shared items), there's a lively discussion going on over there regarding the practical value of recruitment versus assessment practices.
It started with Wendell Williams' first post on how to identify a bad test (the second part is also worth reading). The comments begin relatively benignly, debating the strength of various predictors of performance (e.g., P-O fit versus behavioral interviews), but turns into, let's say, a lively debate that includes a discussion of the Gallup 12, the limitation of assessments, the Uniform Guidelines, and a lot more. The most heated exchange occurs between Wendell and Lou Adler, where accusations and sarcasm fly.
Speaking of Lou, he continues to advocate his perspective with his next post on whether increasing interview accuracy increases quality of hire (yes, he's suggesting that's an open question). While the comments following are fewer in number, the debate continues regarding the value of assessment and the evidence used to support it (e.g., Schmidt & Hunter's 1998 piece).
Who said HR is boring?
Subscribe to:
Posts (Atom)