A few new journal issues have come out lately:
Summer 2016 Personnel Psychology, including:
Transparency of Assessment Centers: Low Criterion-related Validity but Greater Opportunity to Perform?
May 2016 Journal of Applied Psychology, including:
Initial impressions: What they are, what they are not, and how they influence structured interview outcomes.
Racioethnicity, community makeup, and potential employees’ reactions to organizational diversity management approaches.
June 2016 International Journal of Selection and Assessment, including:
Applicant Reactions to Selection Events: Four studies into the role of attributional style and fairness perceptions
Behavioral Cues as Indicators of Deception in Structured Employment Interviews
The Role of Self-focused Attention and Negative Self-thought in Interview Anxiety: A test of two interventions
The Influence of Candidate Social Effectiveness on Assessment Center Performance Ratings: A field study
Discrimination due to Ethnicity and Gender: How susceptible are video-based job interviews?
A Comparison of General and Work-specific Personality Measures as Predictors of Organizational Citizenship Behavior
The Perceived Nature and Incidence of Dysfunctional Assessment Center Features and Processes
Who is Being Judged Promotable: Good actors, high performers, highly committed or birds of a feather?
Showing posts with label Discrimination. Show all posts
Showing posts with label Discrimination. Show all posts
Wednesday, May 04, 2016
Sunday, November 24, 2013
Research update
Well, it's that time of year again. No, not the holidays. No, not winter (or summer, depending on where you are!). Research update time! And I think you will agree with me that there is a lot of interesting research being reported, on traditional topics as well as emerging ones.
First, the November issue of JOB:
- Do transformational leaders increase creative performance and the display of OCBs? Well, that may depend on how much trait affectivity they had to begin with. A reminder to not make blanket statements like "X type of leadership causes Y type of behavior."
- There is seemingly endless debate about the utility of personality inventories. This study reminds us--again--that in assessment research there are few simple answers. The authors describe how a particular combination of personality measures correlated with task performance among professional employees, but not non-professionals. (yes, I said task performance)
Next, the Winter issue of Personnel Psychology (free right now!), much of which is devoted to corporate social responsibility (CSR):
- Do perceptions of CSR drive job pursuit intentions? It may depend on the applicant's previous justice experiences and their moral identity.
- Oh, and it may also depend on the extent to which applicants desire to have an impact through their work.
- There is a debate in the assessment center literature about whether competency dimensions are being measured or if it's purely a function of the assessment type. This study suggests that previous research has been hamstrung by a methodological artifact and that measured properly, assessment centers do in fact assess dimensions.
Let's switch to the November issue of the Journal of Applied Social Psychology:
- Engagement is all the rage, having seemingly displaced the age-old concept of job satisfaction (we'll see). This study reminds us that personality plays an important role in predicting engagement (so by extension our ability to increase engagement may be bounded).
- Here's another good one and it's related to internal motivations. The authors developed an instrument that helps organizations measure the "perception of the extant motivational climate." What does that mean? As I understand it, it's essentially whether most people are judging their performance against their peers or their own internal standards. It seems the latter may result in better results, such as less burnout.
- On to something more closely tied to assessment: letters of recommendation (LORs). There's surprisingly little research on these, but this study adds to our knowledge by suggesting that gender and racial bias can occur in their review, but requiring a more thorough review of them may reduce this (I don't know how likely this is for the average supervisor).
- Finally, a study looking at the evaluation of job applicants who voluntarily interrupted their college attendance. Unfortunately this does not appear to have been perceived as a good thing, and the researchers found a gender bias such that women with interrupted attendance had the lowest evaluations.
Next, the November issue of Industrial and Organizational Psychology, where the second focal article focus on eradicating employment discrimination. This article looks pretty juicy. I haven't received this one yet in the mail, so I may have more to say after digesting it. There are, as always, several commentaries following the focal article, on topics including background checks, childhood differences, and social networks.
Okay, let's tackle the 800-pound gorilla: the December issue of IJSA:
- Are true scores and construct scores the same? According to this Monte Carlo study, it seems how the scales were constructed makes a difference.
- Can non-native accents impact the evaluation of job applicants? Sure seems that way according to this study. But the effect was mediated by similarity, interpersonal attraction, and understandability.
- Here's a fascinating one. A study of applicants for border rangers in the Norwegian Armed Forces showed that psychological hardiness--particularly commitment--predicted completion of a rigorous physical activity above and beyond physical fitness, nutrition, and sensation seeking.
- Psst....recruiters...make sure when you're selling your organization you stay positive.
- Spatial ability. It's a classic KSA that's been studied for a long time, for various reasons including its tie to military assessments and the finding that measures can result in sex differences. But not so fast, spatial ability is not a unitary concept.
- Another study of assessment centers, this time in Russia and using a consensus scoring model.
- And let's round it out with one that should rock some worlds: the authors presents results that suggest that subject matter expert judgment of ability/competency importance bore little relation to test validity! Okay, I'm really curious about what the authors say about the implications, so if anyone reads this one, let us know!
Last but not least, the November issue of the Journal of Applied Psychology:
- Another on personality testing, this one underlining the important distinction between broad and narrow traits. This is another article I'm very curious about.
- Here's on one leadership: specifically, on the impact of different power distance values between leader and subordinates on team effectiveness
- And another on nonnative speakers! This one found discriminatory judgments made against nonnative speakers applying for middle management positions as well as venture funding. Interestingly, it appears to be fully mediated by perceptions of political skill--a topic that is hot right now.
- Okay, let's leave on a big note. This meta-analysis found an improvement in performance prediction of 50% when a mechanical combination of assessment data was used rather than a holistic (judgment-based) method. BOOM! Think about that the next time a hiring supervisor derides your spreadsheet.
Until next time!
First, the November issue of JOB:
- Do transformational leaders increase creative performance and the display of OCBs? Well, that may depend on how much trait affectivity they had to begin with. A reminder to not make blanket statements like "X type of leadership causes Y type of behavior."
- There is seemingly endless debate about the utility of personality inventories. This study reminds us--again--that in assessment research there are few simple answers. The authors describe how a particular combination of personality measures correlated with task performance among professional employees, but not non-professionals. (yes, I said task performance)
Next, the Winter issue of Personnel Psychology (free right now!), much of which is devoted to corporate social responsibility (CSR):
- Do perceptions of CSR drive job pursuit intentions? It may depend on the applicant's previous justice experiences and their moral identity.
- Oh, and it may also depend on the extent to which applicants desire to have an impact through their work.
- There is a debate in the assessment center literature about whether competency dimensions are being measured or if it's purely a function of the assessment type. This study suggests that previous research has been hamstrung by a methodological artifact and that measured properly, assessment centers do in fact assess dimensions.
Let's switch to the November issue of the Journal of Applied Social Psychology:
- Engagement is all the rage, having seemingly displaced the age-old concept of job satisfaction (we'll see). This study reminds us that personality plays an important role in predicting engagement (so by extension our ability to increase engagement may be bounded).
- Here's another good one and it's related to internal motivations. The authors developed an instrument that helps organizations measure the "perception of the extant motivational climate." What does that mean? As I understand it, it's essentially whether most people are judging their performance against their peers or their own internal standards. It seems the latter may result in better results, such as less burnout.
- On to something more closely tied to assessment: letters of recommendation (LORs). There's surprisingly little research on these, but this study adds to our knowledge by suggesting that gender and racial bias can occur in their review, but requiring a more thorough review of them may reduce this (I don't know how likely this is for the average supervisor).
- Finally, a study looking at the evaluation of job applicants who voluntarily interrupted their college attendance. Unfortunately this does not appear to have been perceived as a good thing, and the researchers found a gender bias such that women with interrupted attendance had the lowest evaluations.
Next, the November issue of Industrial and Organizational Psychology, where the second focal article focus on eradicating employment discrimination. This article looks pretty juicy. I haven't received this one yet in the mail, so I may have more to say after digesting it. There are, as always, several commentaries following the focal article, on topics including background checks, childhood differences, and social networks.
Okay, let's tackle the 800-pound gorilla: the December issue of IJSA:
- Are true scores and construct scores the same? According to this Monte Carlo study, it seems how the scales were constructed makes a difference.
- Can non-native accents impact the evaluation of job applicants? Sure seems that way according to this study. But the effect was mediated by similarity, interpersonal attraction, and understandability.
- Here's a fascinating one. A study of applicants for border rangers in the Norwegian Armed Forces showed that psychological hardiness--particularly commitment--predicted completion of a rigorous physical activity above and beyond physical fitness, nutrition, and sensation seeking.
- Psst....recruiters...make sure when you're selling your organization you stay positive.
- Spatial ability. It's a classic KSA that's been studied for a long time, for various reasons including its tie to military assessments and the finding that measures can result in sex differences. But not so fast, spatial ability is not a unitary concept.
- Another study of assessment centers, this time in Russia and using a consensus scoring model.
- And let's round it out with one that should rock some worlds: the authors presents results that suggest that subject matter expert judgment of ability/competency importance bore little relation to test validity! Okay, I'm really curious about what the authors say about the implications, so if anyone reads this one, let us know!
Last but not least, the November issue of the Journal of Applied Psychology:
- Another on personality testing, this one underlining the important distinction between broad and narrow traits. This is another article I'm very curious about.
- Here's on one leadership: specifically, on the impact of different power distance values between leader and subordinates on team effectiveness
- And another on nonnative speakers! This one found discriminatory judgments made against nonnative speakers applying for middle management positions as well as venture funding. Interestingly, it appears to be fully mediated by perceptions of political skill--a topic that is hot right now.
- Okay, let's leave on a big note. This meta-analysis found an improvement in performance prediction of 50% when a mechanical combination of assessment data was used rather than a holistic (judgment-based) method. BOOM! Think about that the next time a hiring supervisor derides your spreadsheet.
Until next time!
Thursday, June 13, 2013
Another (smaller) research update
Seems like this always happens, I post a research update and then Personnel Psychology comes out with their latest issue.
Oh well! At least you won't have to read as much this time And while we're at it, let's take a look at the latest Journal of Applied Social Psychology. But first, Personnel Psych:
- The relationship between task performance and citizenship behavior: more complicated than you might think
- Propensity scoring: a statistical technique that may improve our ability to make causal inferences from quasi-experimental designs
- Can creative employees drive higher organizational performance? Well that seems to depend on how open to risk your organization is...
- It's easier said than done, but those looking for a job should try to stay positive: it influences how quickly you're likely to land your next position
- Last but not least, a small correction to a study posted earlier on the advantage of contextualizing personality inventory items
Okay, on to June 2013 JASP, which is a special issue on prejudice:
-Trying to reduce bias in your selection process? Make sure your raters get enough sleep.
- Discrimination against people based on their weight: it starts early! (there's another study on weight discrimination that looked at its impact on perceived social status)
- Discrimination against men perceived to be Muslim increased among Western observers when subjects were presented in traditional dress.
- Those of Black/White biracial descent were less likely to be perceived as minorities and thus less appropriate receipts of affirmative action
- Okay, I like this one in particular, and it happens to be the last one I'll describe. Researchers showed that competency perceptions of an African-American man hired varied depending on whether it was under "affirmative action" or a "diversity initiative"--the latter resulting in higher ratings. This was particularly true for conservative, White raters. Just goes to show how important words are, as any political consultant will tell you.
Fin.
Oh well! At least you won't have to read as much this time And while we're at it, let's take a look at the latest Journal of Applied Social Psychology. But first, Personnel Psych:
- The relationship between task performance and citizenship behavior: more complicated than you might think
- Propensity scoring: a statistical technique that may improve our ability to make causal inferences from quasi-experimental designs
- Can creative employees drive higher organizational performance? Well that seems to depend on how open to risk your organization is...
- It's easier said than done, but those looking for a job should try to stay positive: it influences how quickly you're likely to land your next position
- Last but not least, a small correction to a study posted earlier on the advantage of contextualizing personality inventory items
Okay, on to June 2013 JASP, which is a special issue on prejudice:
-Trying to reduce bias in your selection process? Make sure your raters get enough sleep.
- Discrimination against people based on their weight: it starts early! (there's another study on weight discrimination that looked at its impact on perceived social status)
- Discrimination against men perceived to be Muslim increased among Western observers when subjects were presented in traditional dress.
- Those of Black/White biracial descent were less likely to be perceived as minorities and thus less appropriate receipts of affirmative action
- Okay, I like this one in particular, and it happens to be the last one I'll describe. Researchers showed that competency perceptions of an African-American man hired varied depending on whether it was under "affirmative action" or a "diversity initiative"--the latter resulting in higher ratings. This was particularly true for conservative, White raters. Just goes to show how important words are, as any political consultant will tell you.
Fin.
Sunday, May 19, 2013
Research update
Okay, I'm just gonna say it. There is an insane amount of research out there directly or indirectly related to our field. I'll be honest, it's a little daunting thinking about reading and processing all of it.
Luckily, you have me to overly simplify it for you so you can incorrectly describe it to others. So let's jump right in!
First up, the June issue of IJSA:
- Fairness perceptions matter in web-based selection too!
- The predictive validity of conscientiousness is moderated by self-enhancement
- Speaking of conscientiousness, it's a better predictor of performance in routinized jobs than those with complex cognitive requirements
- Psychological hardiness predicts adaptability in military leaders. This study is awesome also for showing a negative relationship between SAT scores and West Point performance.
- Recruiters, listen up: this study found that contingent workers converted to full-time status performed as well as referral and online hires, but not as well as those sourced internally (bookmark this one, there's a dearth of recruitment research)
- Scoring biodata: empirical keying reigns supreme over rational and quasi-rational approaches. Quasi-rational? Sounds like me most of the time.
- Personality and job performance: interactions are important beyond main effects
Okay, next, the May issue of JAP:
- Adaptive decision making in military leaders: both the brain and the mind are important (okay you monists out there, chill out)
- Are you a MANOVA (wo)man? Then read this.
- Proactive leaders set more challenging goals and have higher sales performance--assuming they have the trust of their subordinates.
Next, the April issue of JOB. Just one study, albeit interesting, in which female evaluators were less likely to recommend hiring or promoting Asian (versus White) applicants into jobs requiring social skills
Speaking of JOB, how about the May issue?
- Have you been calling for more research on calling? Your wish is granted.
- Curious about the concept of curiosity? (okay, I'll stop) Looks like it can be predictive of job performance above and beyond traditional cognitive and non-cognitive predictors. I'm gonna assume it varies with job, but I think the authors are probably right that it will increase in importance over time.
- When selecting for teams that may experience crisis situations (e.g., nuclear power plant crews), consider homogeneity--not mean levels--of positive affect.
- Moderate levels of supervisory structure combined with high levels of consideration lead to lowest levels of CWB's
- The dark side of OCBs
- The light side of OCBs
- Dark side and light side working together in a picture that has nothing to do with selection
Okay, onwards and upwards: one from the April issue of JASP, on generalized self-efficacy, work-related self-efficacy, and job-related outcomes
And one from the May issue: Sensation seeking and need for structure predict military field exercise performance
How about a little EI research? Haven't had a lot of that lately. Here's a piece from the May issue of Journal of Management that found emotion management ability to be a valid predictor of job performance.
Let's look in the May Psychological Science:
- Where we find a fascinating study that demonstrates assessment of profound cognitive abilities at a young age predicts outstanding contributions in many adults
- Another interesting study of how an individual responds to daily stressors (which I would posit are differentially found in various jobs) has long-term consequences for their mental health
- High schoolers with high math and verbal skills are less likely to choose STEM careers than those with math skills but moderate verbal skills
- Okay, check this one out: mindfulness training improved GRE reading comp scores as well as working memory. Implications for personnel assessment??
Shifting to the May Psychological Bulletin:
- Rorschach, anyone?
- Believing that individuals are malleable, rather than fixed ("implicit theory") predicts goal achievement. Implications for leaders?
Last but not least, let's hook up with our friends at PARE and see what they're up to:
- Into Excel? Check out this article on forest plots
- Or maybe factor analysis is your thing.
That's all folks! Hope your brain is spinning like mine!
Luckily, you have me to overly simplify it for you so you can incorrectly describe it to others. So let's jump right in!
First up, the June issue of IJSA:
- Fairness perceptions matter in web-based selection too!
- The predictive validity of conscientiousness is moderated by self-enhancement
- Speaking of conscientiousness, it's a better predictor of performance in routinized jobs than those with complex cognitive requirements
- Psychological hardiness predicts adaptability in military leaders. This study is awesome also for showing a negative relationship between SAT scores and West Point performance.
- Recruiters, listen up: this study found that contingent workers converted to full-time status performed as well as referral and online hires, but not as well as those sourced internally (bookmark this one, there's a dearth of recruitment research)
- Scoring biodata: empirical keying reigns supreme over rational and quasi-rational approaches. Quasi-rational? Sounds like me most of the time.
- Personality and job performance: interactions are important beyond main effects
Okay, next, the May issue of JAP:
- Adaptive decision making in military leaders: both the brain and the mind are important (okay you monists out there, chill out)
- Are you a MANOVA (wo)man? Then read this.
- Proactive leaders set more challenging goals and have higher sales performance--assuming they have the trust of their subordinates.
Next, the April issue of JOB. Just one study, albeit interesting, in which female evaluators were less likely to recommend hiring or promoting Asian (versus White) applicants into jobs requiring social skills
Speaking of JOB, how about the May issue?
- Have you been calling for more research on calling? Your wish is granted.
- Curious about the concept of curiosity? (okay, I'll stop) Looks like it can be predictive of job performance above and beyond traditional cognitive and non-cognitive predictors. I'm gonna assume it varies with job, but I think the authors are probably right that it will increase in importance over time.
- When selecting for teams that may experience crisis situations (e.g., nuclear power plant crews), consider homogeneity--not mean levels--of positive affect.
- Moderate levels of supervisory structure combined with high levels of consideration lead to lowest levels of CWB's
- The dark side of OCBs
- The light side of OCBs
- Dark side and light side working together in a picture that has nothing to do with selection
Okay, onwards and upwards: one from the April issue of JASP, on generalized self-efficacy, work-related self-efficacy, and job-related outcomes
And one from the May issue: Sensation seeking and need for structure predict military field exercise performance
How about a little EI research? Haven't had a lot of that lately. Here's a piece from the May issue of Journal of Management that found emotion management ability to be a valid predictor of job performance.
Let's look in the May Psychological Science:
- Where we find a fascinating study that demonstrates assessment of profound cognitive abilities at a young age predicts outstanding contributions in many adults
- Another interesting study of how an individual responds to daily stressors (which I would posit are differentially found in various jobs) has long-term consequences for their mental health
- High schoolers with high math and verbal skills are less likely to choose STEM careers than those with math skills but moderate verbal skills
- Okay, check this one out: mindfulness training improved GRE reading comp scores as well as working memory. Implications for personnel assessment??
Shifting to the May Psychological Bulletin:
- Rorschach, anyone?
- Believing that individuals are malleable, rather than fixed ("implicit theory") predicts goal achievement. Implications for leaders?
Last but not least, let's hook up with our friends at PARE and see what they're up to:
- Into Excel? Check out this article on forest plots
- Or maybe factor analysis is your thing.
That's all folks! Hope your brain is spinning like mine!
Saturday, February 09, 2013
Research update
It's that time, February research update--and there are some really fascinating studies out there:
First the March 2013 issue of IJSA:
- Procedural fairness is an important aspect of organizational attraction, especially for rejected applicants.
- Does personality type impact perceptions of the selection process? According to this study and their measures (Bohemian is a personality type?), sometimes.
- Do you use SEM to generate a SED and ultimately score bands? Then you will definitely want to read this piece, which suggests the traditional method is flawed.
- Emotional intelligence continues to be a hot topic. In this study the authors describe the development of a SJT-based measure.
- Staying on the topic of personality, this study adds to the debate over the utility of constructs vs. facets in predicting performance--this time with conscientiousness predicting police officer performance.
- P-O fit is often used to explain why people apply for certain jobs. In this study researchers found that values that applicants find attractive are particularly important (as opposed to neutral or aversive aspects).
- I've written about automated reference check systems before. In fact I've written about the company that the first author of this study is from (SkillSurvey). What is partially a demonstration of the value of their product also adds to our knowledge of reference checks by finding no demographic differences and the ability to predict involuntary (but not voluntary) turnover.
- How big of a deal is it when it is discovered that an applicant "embellished" on their resume? Depends on how much you already liked them.
- In this study of applicants in Iran, the authors found that the usefulness of a web-based application system was more important than how easy it was to use. So basically looks are important but not sufficient.
Next, the January issue of the Journal of Applied Psychology:
- Construct validity has always been a sticky wicket. The authors of this study take an interesting approach, using video-based testing to illustrate a method for supporting construct validity.
- This study explores the variance in assessment center ratings stemming from various sources, and the authors argue that existing literature has masked this variance.
-The authors of this study show how the application of a particular bootstrap method applied to meta-analyses increases accuracy.
- Speaking of meta-analysis, the authors of this study demonstrate that the differences between coefficient alpha and composite reliability in actual data sets is quite small.
On to the January issue of the Journal of Applied Social Psychology, which as of the time of this post was available for free!
- Similar to findings in other areas, the impact of negative publicity is stronger than positive publicity on organizational attraction, according to the authors of this study. The authors also found in these cases, organizations would do well to provide detailed recruiting ads to mitigate the impact.
- Age discrimination seems to be increasing in popularity as a research topic (which makes sense from a demographic perspective). In this study, the authors found raters discriminated against older and younger applicants, with the former being the least likely to be hired.
- Here's an interesting one: in this study, people of color perceived White women to be, among other things, ditsy, shallow, privileged, and appearance focused. Stereotypes know no color boundaries.
Let's take a look at the February 2013 issue of the Journal of Personality and Social Psychology:
- First, a fascinating look at how generalizable the five-factor model of personality is across cultures. Specifically, the authors looked at a largely illiterate, indigenous culture in Bolivia. They failed to find support for the widely used model, and suggest that the structure of personality may differ across societies.
- Second, a study of sex and gender differences that points out the importance of considering the construct of gender (as opposed to the binary concept of sex) for a variety of behavioral and psychological indicators, including big five personality.
Let's move now to the February 2013 issue of Psychological Science, where:
- In this study, the authors found that age differences in cognitive functioning within cohorts were often as large as those between cohorts, which calls into question the cohort explanation for age differences.
- In a study that I think has implications for personnel assessment, the authors found that raters of M.B.A. applications that had already given several high recommendations were less likely to do so later in the day. Another bias to watch out for!
Last but not least, there was an article recently in the New York Times about how important referrals are for some organizations and the increased importance this places on social networking sites. There was a reference to a study done in August of 2012 on source of hire that I hadn't seen before, which found that not only were referrals more likely to be hired, they had longer tenure--which is consistent with other research I've seen. Unfortunately, no word about performance differences.
That's all for now!
First the March 2013 issue of IJSA:
- Procedural fairness is an important aspect of organizational attraction, especially for rejected applicants.
- Does personality type impact perceptions of the selection process? According to this study and their measures (Bohemian is a personality type?), sometimes.
- Do you use SEM to generate a SED and ultimately score bands? Then you will definitely want to read this piece, which suggests the traditional method is flawed.
- Emotional intelligence continues to be a hot topic. In this study the authors describe the development of a SJT-based measure.
- Staying on the topic of personality, this study adds to the debate over the utility of constructs vs. facets in predicting performance--this time with conscientiousness predicting police officer performance.
- P-O fit is often used to explain why people apply for certain jobs. In this study researchers found that values that applicants find attractive are particularly important (as opposed to neutral or aversive aspects).
- I've written about automated reference check systems before. In fact I've written about the company that the first author of this study is from (SkillSurvey). What is partially a demonstration of the value of their product also adds to our knowledge of reference checks by finding no demographic differences and the ability to predict involuntary (but not voluntary) turnover.
- How big of a deal is it when it is discovered that an applicant "embellished" on their resume? Depends on how much you already liked them.
- In this study of applicants in Iran, the authors found that the usefulness of a web-based application system was more important than how easy it was to use. So basically looks are important but not sufficient.
Next, the January issue of the Journal of Applied Psychology:
- Construct validity has always been a sticky wicket. The authors of this study take an interesting approach, using video-based testing to illustrate a method for supporting construct validity.
- This study explores the variance in assessment center ratings stemming from various sources, and the authors argue that existing literature has masked this variance.
-The authors of this study show how the application of a particular bootstrap method applied to meta-analyses increases accuracy.
- Speaking of meta-analysis, the authors of this study demonstrate that the differences between coefficient alpha and composite reliability in actual data sets is quite small.
On to the January issue of the Journal of Applied Social Psychology, which as of the time of this post was available for free!
- Similar to findings in other areas, the impact of negative publicity is stronger than positive publicity on organizational attraction, according to the authors of this study. The authors also found in these cases, organizations would do well to provide detailed recruiting ads to mitigate the impact.
- Age discrimination seems to be increasing in popularity as a research topic (which makes sense from a demographic perspective). In this study, the authors found raters discriminated against older and younger applicants, with the former being the least likely to be hired.
- Here's an interesting one: in this study, people of color perceived White women to be, among other things, ditsy, shallow, privileged, and appearance focused. Stereotypes know no color boundaries.
Let's take a look at the February 2013 issue of the Journal of Personality and Social Psychology:
- First, a fascinating look at how generalizable the five-factor model of personality is across cultures. Specifically, the authors looked at a largely illiterate, indigenous culture in Bolivia. They failed to find support for the widely used model, and suggest that the structure of personality may differ across societies.
- Second, a study of sex and gender differences that points out the importance of considering the construct of gender (as opposed to the binary concept of sex) for a variety of behavioral and psychological indicators, including big five personality.
Let's move now to the February 2013 issue of Psychological Science, where:
- In this study, the authors found that age differences in cognitive functioning within cohorts were often as large as those between cohorts, which calls into question the cohort explanation for age differences.
- In a study that I think has implications for personnel assessment, the authors found that raters of M.B.A. applications that had already given several high recommendations were less likely to do so later in the day. Another bias to watch out for!
Last but not least, there was an article recently in the New York Times about how important referrals are for some organizations and the increased importance this places on social networking sites. There was a reference to a study done in August of 2012 on source of hire that I hadn't seen before, which found that not only were referrals more likely to be hired, they had longer tenure--which is consistent with other research I've seen. Unfortunately, no word about performance differences.
That's all for now!
Sunday, September 30, 2012
Research update: September, 2012
It's been a while since I've posted a research update. I was waiting for a critical mass but honestly it was taking a while! But no more waiting, let's see what's out there...
First up, a topic near and dear to my heart: HR certification. In the September/October issue of HRM, Lyons, et al. took a look at a sample of web-based HR job ads to study the prevalence of requiring the PHR/SPHR certifications offered by SHRM, er, the the Human Resource Certification Institute. Back in 2005, a similar study found that less than 2 percent of jobs listed the certifications as preferred or required. This time? 15.6 percent. Boom! Quite an increase (although obviously still far from the majority). Now the important question: does possession of said certificates actually predict job performance? I gotta be totally honest, I've seen some blind admiration of these certifications without any indication that this question was addressed.
Anyhoo, what else is in that issue of HRM? For one, a study of perceived supervisor support and team-level performance. Subjects: 75 gas stations in Norway (I just wanted to say that). Results? Link between the two. Implication? Another competency to consider when hiring supervisors. Oh, and btw, there's another study in the same issue about HR practices in the organization and links to OCB and customer service (hint: participation is good).
Like soccer (or, should I say, football)? How about team research? Either way you'll be interested in this fascinating study, which found that players transferred to another team improved the performance of their new team against their old one.
Finally, this small study which found having applicants complete self-affirming written self-guidance statements prior to interviewing improved their performance.
Okay, next up is the September issue of JAP.
How about another leadership study? Why not. In this one, researchers found supervisor consideration behaviors were positively related to employee attitudes (and apparently the more the merrier) while there was an ideal level of initiating structure behavior. There's a lot more with this one, particularly on P-E fit, so check it out.
Have you ever found yourself wondering, "I wonder what the power of cross-level interaction effects is when conducting tests of multilevel contingency and interactionism"? Well today's your lucky day, because check this out.
Promoting and fostering diversity in organizations is hypothesized to have various positive outcomes, one of which is creativity. But previous results are mixed. In this lab study, the authors propose an explanation: it depends on the extent to which team members take each others' perspectives. Another reminder that simply recruiting and hiring people with different backgrounds does not ensure successful performance.
Anyone performing differential item functioning (DIF) analysis should check out this study, which recommends a particular procedure for identifying optimal anchor items.
Finally, and this doesn't directly relate to assessment but is just darn interesting, a study on "cyberloafing" and the impact that moving to daylight savings time has on it (hint: it's not good).
Okay, back to the topic at hand and the October JOB.
Need another example of the complexity of the relationship between personality factors and organizational behavior? Check this out (personality factor: conscientiousness).
Speaking of conscientiousness, this study found that adaptive performance (acquiring new competencies as a result of organizational change) was related to task performance but the relationship was impacted by employee conscientiousness (and organizational politics!).
Did someone say conscientiousness? No, seriously, another study on it. But it's another interesting one that looks at the importance of group perceptions as well as team composition when analyzing the relationship between individual personality measures and performance. And it's another sports team study, this time university football players (U.S. football that is).
Last but not least, the October issue of JPSP. And the first study may just be the most interesting thing I've seen in a while.
In it, the authors demonstrate via lab and field experiments with a variety of subjects and decisions (including graduate school admission), that people seem to have a preference for potential rather than achievement. What are the implications for recruiting and assessment? That decision makers may be unduly influenced (at the cost of validity) by the promise of an applicant rather than their demonstrated accomplishments. However, this warning must be moderated with an acknowledgement that sometimes potential (e.g., ability scores) can be equally valid sources of predictive validity. So, bottom line: another bias to watch out for.
Next, an elucidating if slightly depressing study of stereotype threat. The authors demonstrate that the experience of stereotype threat among African Americans and Hispanic/Latino(a)s resulted in scientific disidentification and intention to pursue a scientific career.
Okay, this next one is tricky because it looks interesting but I wasn't able to find an in-press version (maybe a helpful reader can point one out?). It's a refinement of a theory of basic individual values, and looks like it has implications for career and applicant selection.
Finally, very last but most definitely not least, a study of the accuracy of personality judgments, a particularly timely topic given recent research that suggests these judgments have significant value in personnel assessment. Specifically, the authors were looking at hindsight effects (essentially how your perception changes after time and additional information). Good stuff and implications for anyone wanting to use observer measures of personality.
First up, a topic near and dear to my heart: HR certification. In the September/October issue of HRM, Lyons, et al. took a look at a sample of web-based HR job ads to study the prevalence of requiring the PHR/SPHR certifications offered by SHRM, er, the the Human Resource Certification Institute. Back in 2005, a similar study found that less than 2 percent of jobs listed the certifications as preferred or required. This time? 15.6 percent. Boom! Quite an increase (although obviously still far from the majority). Now the important question: does possession of said certificates actually predict job performance? I gotta be totally honest, I've seen some blind admiration of these certifications without any indication that this question was addressed.
Anyhoo, what else is in that issue of HRM? For one, a study of perceived supervisor support and team-level performance. Subjects: 75 gas stations in Norway (I just wanted to say that). Results? Link between the two. Implication? Another competency to consider when hiring supervisors. Oh, and btw, there's another study in the same issue about HR practices in the organization and links to OCB and customer service (hint: participation is good).
Like soccer (or, should I say, football)? How about team research? Either way you'll be interested in this fascinating study, which found that players transferred to another team improved the performance of their new team against their old one.
Finally, this small study which found having applicants complete self-affirming written self-guidance statements prior to interviewing improved their performance.
Okay, next up is the September issue of JAP.
How about another leadership study? Why not. In this one, researchers found supervisor consideration behaviors were positively related to employee attitudes (and apparently the more the merrier) while there was an ideal level of initiating structure behavior. There's a lot more with this one, particularly on P-E fit, so check it out.
Have you ever found yourself wondering, "I wonder what the power of cross-level interaction effects is when conducting tests of multilevel contingency and interactionism"? Well today's your lucky day, because check this out.
Promoting and fostering diversity in organizations is hypothesized to have various positive outcomes, one of which is creativity. But previous results are mixed. In this lab study, the authors propose an explanation: it depends on the extent to which team members take each others' perspectives. Another reminder that simply recruiting and hiring people with different backgrounds does not ensure successful performance.
Anyone performing differential item functioning (DIF) analysis should check out this study, which recommends a particular procedure for identifying optimal anchor items.
Finally, and this doesn't directly relate to assessment but is just darn interesting, a study on "cyberloafing" and the impact that moving to daylight savings time has on it (hint: it's not good).
Okay, back to the topic at hand and the October JOB.
Need another example of the complexity of the relationship between personality factors and organizational behavior? Check this out (personality factor: conscientiousness).
Speaking of conscientiousness, this study found that adaptive performance (acquiring new competencies as a result of organizational change) was related to task performance but the relationship was impacted by employee conscientiousness (and organizational politics!).
Did someone say conscientiousness? No, seriously, another study on it. But it's another interesting one that looks at the importance of group perceptions as well as team composition when analyzing the relationship between individual personality measures and performance. And it's another sports team study, this time university football players (U.S. football that is).
Last but not least, the October issue of JPSP. And the first study may just be the most interesting thing I've seen in a while.
In it, the authors demonstrate via lab and field experiments with a variety of subjects and decisions (including graduate school admission), that people seem to have a preference for potential rather than achievement. What are the implications for recruiting and assessment? That decision makers may be unduly influenced (at the cost of validity) by the promise of an applicant rather than their demonstrated accomplishments. However, this warning must be moderated with an acknowledgement that sometimes potential (e.g., ability scores) can be equally valid sources of predictive validity. So, bottom line: another bias to watch out for.
Next, an elucidating if slightly depressing study of stereotype threat. The authors demonstrate that the experience of stereotype threat among African Americans and Hispanic/Latino(a)s resulted in scientific disidentification and intention to pursue a scientific career.
Okay, this next one is tricky because it looks interesting but I wasn't able to find an in-press version (maybe a helpful reader can point one out?). It's a refinement of a theory of basic individual values, and looks like it has implications for career and applicant selection.
Finally, very last but most definitely not least, a study of the accuracy of personality judgments, a particularly timely topic given recent research that suggests these judgments have significant value in personnel assessment. Specifically, the authors were looking at hindsight effects (essentially how your perception changes after time and additional information). Good stuff and implications for anyone wanting to use observer measures of personality.
Sunday, April 01, 2012
April research update
Okay, so I didn't quite hit my March update...date. But I'm awful close, so without further ado let's take a look at what research has come out lately. And boy is there a lot to cover...
First, the Spring Personnel Psychology, which it should be noted is all free at the time of this writing (!):
- Moore, et al. describe the development and test of a measure of "an individual's propensity to morally disengage", which is really (as the title suggests) about investigating why people do bad things at work. Looks like it has promise beyond existing measures that predict things like CWBs.
- Next, O'Boyle and Aguinis present the results of a fascinating study of the distribution of human performance. Turns out it may not be normal after all, but rather Paretian. This has big implications for...well, all sorts of things, including selection. Read here for more.
- Avery, et al. describe results of a study of racioethnic matching (employees and customers) and the impact on productivity (which turned out to be positive, through customer satisfaction). The authors present several very practical ways of interpreting this finding without jumping to hiring based on race.
Next, the March Journal of Applied Psychology:
- First, van Hooft and Born with a fascinating study of eye-tracking to investigating faking on personality and integrity measures. Looks like eye movement differs depending on the intent to inflate, and it also suggests response time could be an indicator of inflation.
- Next, Madera and Hebl with another eye-tracking study, but this time on the impact that facial stigmatization has on interview performance. Discouragingly (but perhaps not surprising), the results suggest individuals with facial stigmatization may receive lower ratings, in part due to the interviewer being distracted.
- Into core self-evaluations? You might want to read this study by Wu and Griffin, in which they argue that CSEs are predictors of, but also influenced by, contextual factors such as job satisfaction.
- Lievens and Sackett provide evidence that individuals' procedural knowledge of interpersonal behavior may be valuable in predicting performance (in this case, medical students).
- Bernerth and colleagues discuss the usefulness of credit scores in predicting job performance, which I wrote about in an earlier blog post.
Next up, the May Journal of Organizational Behavior:
- Peng, et al. suggest that deployed soldiers with higher levels of conscientiousness and lower levels of neuroticism may be able to better cope with psychological distress (the effect of extraversion was mixed).
- Derous, et al. discuss discrimination in resume screening among individuals who belong to multiple minority groups (in this case with a focus on Arab women). Applicant, job, and recruiter characteristics were all important.
The March Industrial and Organizational Psychology has a fascinating focal article on how I/O psychology and HR can contribute to organizational strategy and competitive advantage. Several commentaries follow, at least one of which directly addresses selection. Move quick, because right now both the focal article and the commentaries are free!
Now here are some miscellaneous articles you may be interested in:
- Jackson, et al. describe a study that suggests going through military training may alter someone's personality...
- Using a 1- or 2-item measure of personality traits, thinking you don't need more than that? Think again.
- Looking for creative thinkers? Gino and Ariely provide evidence that may give you pause: those that are the most creative may be more dishonest...(!)
- Hiring people into a particularly political environment (naw, none of us ever have that)? Chang, et al. present results that suggest you may want to pay attention to their self-monitoring skill and level of conscientiousness...but maybe not in the way you think.
- Still getting over St. Patrick's Day? Then check out van den Born and van Witteloostuijn's research on "shamrock" organizations. They suggest this type of organization may explain the conflicting findings on modern job tenure.
Finally, all the presentations from the 2011 IPAC conference that were previously available only to members have been made public. There is so much good stuff here I can't even begin to summarize it. Just go check it out.
First, the Spring Personnel Psychology, which it should be noted is all free at the time of this writing (!):
- Moore, et al. describe the development and test of a measure of "an individual's propensity to morally disengage", which is really (as the title suggests) about investigating why people do bad things at work. Looks like it has promise beyond existing measures that predict things like CWBs.
- Next, O'Boyle and Aguinis present the results of a fascinating study of the distribution of human performance. Turns out it may not be normal after all, but rather Paretian. This has big implications for...well, all sorts of things, including selection. Read here for more.
- Avery, et al. describe results of a study of racioethnic matching (employees and customers) and the impact on productivity (which turned out to be positive, through customer satisfaction). The authors present several very practical ways of interpreting this finding without jumping to hiring based on race.
Next, the March Journal of Applied Psychology:
- First, van Hooft and Born with a fascinating study of eye-tracking to investigating faking on personality and integrity measures. Looks like eye movement differs depending on the intent to inflate, and it also suggests response time could be an indicator of inflation.
- Next, Madera and Hebl with another eye-tracking study, but this time on the impact that facial stigmatization has on interview performance. Discouragingly (but perhaps not surprising), the results suggest individuals with facial stigmatization may receive lower ratings, in part due to the interviewer being distracted.
- Into core self-evaluations? You might want to read this study by Wu and Griffin, in which they argue that CSEs are predictors of, but also influenced by, contextual factors such as job satisfaction.
- Lievens and Sackett provide evidence that individuals' procedural knowledge of interpersonal behavior may be valuable in predicting performance (in this case, medical students).
- Bernerth and colleagues discuss the usefulness of credit scores in predicting job performance, which I wrote about in an earlier blog post.
Next up, the May Journal of Organizational Behavior:
- Peng, et al. suggest that deployed soldiers with higher levels of conscientiousness and lower levels of neuroticism may be able to better cope with psychological distress (the effect of extraversion was mixed).
- Derous, et al. discuss discrimination in resume screening among individuals who belong to multiple minority groups (in this case with a focus on Arab women). Applicant, job, and recruiter characteristics were all important.
The March Industrial and Organizational Psychology has a fascinating focal article on how I/O psychology and HR can contribute to organizational strategy and competitive advantage. Several commentaries follow, at least one of which directly addresses selection. Move quick, because right now both the focal article and the commentaries are free!
Now here are some miscellaneous articles you may be interested in:
- Jackson, et al. describe a study that suggests going through military training may alter someone's personality...
- Using a 1- or 2-item measure of personality traits, thinking you don't need more than that? Think again.
- Looking for creative thinkers? Gino and Ariely provide evidence that may give you pause: those that are the most creative may be more dishonest...(!)
- Hiring people into a particularly political environment (naw, none of us ever have that)? Chang, et al. present results that suggest you may want to pay attention to their self-monitoring skill and level of conscientiousness...but maybe not in the way you think.
- Still getting over St. Patrick's Day? Then check out van den Born and van Witteloostuijn's research on "shamrock" organizations. They suggest this type of organization may explain the conflicting findings on modern job tenure.
Finally, all the presentations from the 2011 IPAC conference that were previously available only to members have been made public. There is so much good stuff here I can't even begin to summarize it. Just go check it out.
Tuesday, December 27, 2011
Final research update of 2011
Welcome to the last HR Tests research update of 2011! This year has been filled with research developments, building on previous thinking as well as venturing out into new areas. Let's see what the end of the year brings us (hint: a lot)...
We'll start with the Winter 2011 issue of Personnel Psychology:
- If you're looking for executives who excel at strategic thinking, you'll want to pay attention to not only their cognitive ability but their accumulated work history, according to Dragoni, et al.
- I tend to think of realistic job previews (RJPs) as occurring pre-hire, but research by Earnest, et al. suggests that an effective technique is to conduct an oral and written RJP post-hire--and be as honest as possible.
- Meta-analyses are relied upon heavily as summaries of large bodies of research. Roth, et al. point out ways we can make them better, particularly with respect to estimates of validity and group differences.
- One of the downsides of cognitive ability tests is they're not always perceived very well by applicants. Sumanth & Cable show how this perception is influenced by the status of the organization as well as individual status.
- Next, a fascinating description of a scale to measure the idea of calling by Dobrow and Tosti-Kharas. What I found most interesting was how the measure was associated with different criteria for different domains (e.g., art, music, management). You can read a draft version here.
Let's turn now to the January issue of JOB:
- Political skill is a hot topic, and Gentry, et al. demonstrate how the perception of promotability related to this skill varies between bosses, peers, and direct reports. Long story short: it differs, and also depends on behavior (attitude will only take you so far, right?). You can read the full version (at least right now!) here.
- Anyone that's worked with (or has in their family) engineers or scientists knows that they often share some strong traits. This suggests leading these groups that engage in creative activity may require specific attributes, which is what Robledo, et al. set out to describe.
On to the January issue of JOM:
- Core self-evaluation (CSE) is another hot topic, and Chang, et al. provide an important review of 15 years of research, including the meta-analytic support for CSE predicting in-role and extra-role performance.
- Boswell, et al. provide an integrative review of the concept of job search across different situations (e.g., following job loss, while employed).
- One area that deserves more attention is selection into the highest positions within organizations. Withers, et al. provide a review of the process of selecting a director for a board.
Let's not forget v41(2) of Personnel Review:
- Hoglund delves into the topic of talent management as a strategic HRM practice. A fascinating topic that reinforces the importance of HRM as an influence over employee perceptions and behavior.
- The wording of job ads can have important impacts on applicant perceptions and behavior. De Cooman and Pepermans analyze the differences between for-profit and non-profit job ads, and show how only a fraction of the information potentially relevant for job-person match is published.
- Another topic that deserves additional attention is the motivation to expatriate. Altman and Baruch describe results of a qualitative study that may be useful to organizations thinking about attracting and selecting for positions that require this substantial move.
Don't forget this one:
- Johnson, et al. with an important reminder that when looking at the issue of discrimination, using single categories to define groups is probably not the best strategy.
Whew! And last but certainly not least, in the December issue of IOP, Michael McDaniel and colleagues present an argument (similar to ones made elsewhere) that the Uniform Guidelines are outdated and, worse than that, a detriment to the field of selection. The commentaries are many and range from support to passionate disagreement, with a healthy dose of caution (and dare I say...intransigence?) thrown in. Worth a read, particularly for those following the Guidelines literally and those engaged in related litigation. You can read a draft version here.
I hope everyone has a great New Years; here's to a wonderful 2012!
Saturday, August 27, 2011
August Research update
Okay, past time for a research update.
- The September issue of IJSA has several articles on job discrimination, including pieces by Anderson, Anseel, and Patterson & Zibarras. Looks like the focus is not only on actual job discrimination but perceived job discrimination.
- Krause, et al. describe assessment center practices in South Africa
- Ree and Carretta get serious about incremental validity versus unique prediction
- What do you need to be a top notch recruiter? Well if your measure of job performance is revenue, don't dismiss emotional intelligence, according to Downey, et al.
- Chen, et al. help us understand how applicant personality interacts with impression management tactics to influence interviewer perceptions
- What type of student succeeds at a campus recruitment drive? Gokuladas studied nearly 600 engineering graduates in South India and found that success correlated with engineering GPA and English proficiency. Interestingly female graduates outperformed their male counterparts for jobs in the software services industry.
- Trying to predict counterproductive work behaviors? Bowling et al. suggest that the interaction between conscientiousness, agreeableness, and neuroticism is important and complex.
- One of the focal articles in the September 2011 Industrial and Organizational Psychology is all about individual psychological assessment--something I don't write about very often. Silzer and Jeanneret focus on executive assessment but cover a lot of ground on the broader topic, including why IPA has its critics and how we can advance the science. It is, as with all the focal articles, followed by several commentaries.
- Son Hing, et al. present a fascinating study of meritocracy and how it differs from other hierarchy-legitimizing ideologies
- Stats fans: Bonett and Wright present a method for obtaining the proper sample size for obtaining a certain confidence interval when conducting multiple regression.
- van Vianen, et al. describe a study that contributes to our understanding of the important of person-organization and person-supervisor fit
Sorry there haven't been more updates lately. New baby + potential layoffs at work = little time for blogging!
- The September issue of IJSA has several articles on job discrimination, including pieces by Anderson, Anseel, and Patterson & Zibarras. Looks like the focus is not only on actual job discrimination but perceived job discrimination.
- Krause, et al. describe assessment center practices in South Africa
- Ree and Carretta get serious about incremental validity versus unique prediction
- What do you need to be a top notch recruiter? Well if your measure of job performance is revenue, don't dismiss emotional intelligence, according to Downey, et al.
- Chen, et al. help us understand how applicant personality interacts with impression management tactics to influence interviewer perceptions
- What type of student succeeds at a campus recruitment drive? Gokuladas studied nearly 600 engineering graduates in South India and found that success correlated with engineering GPA and English proficiency. Interestingly female graduates outperformed their male counterparts for jobs in the software services industry.
- Trying to predict counterproductive work behaviors? Bowling et al. suggest that the interaction between conscientiousness, agreeableness, and neuroticism is important and complex.
- One of the focal articles in the September 2011 Industrial and Organizational Psychology is all about individual psychological assessment--something I don't write about very often. Silzer and Jeanneret focus on executive assessment but cover a lot of ground on the broader topic, including why IPA has its critics and how we can advance the science. It is, as with all the focal articles, followed by several commentaries.
- Son Hing, et al. present a fascinating study of meritocracy and how it differs from other hierarchy-legitimizing ideologies
- Stats fans: Bonett and Wright present a method for obtaining the proper sample size for obtaining a certain confidence interval when conducting multiple regression.
- van Vianen, et al. describe a study that contributes to our understanding of the important of person-organization and person-supervisor fit
Sorry there haven't been more updates lately. New baby + potential layoffs at work = little time for blogging!
Saturday, July 23, 2011
July Research Update
Okay, it's past time for a research update. Let's see what's going on out there:
- Looking to get a better salary offer? Research by Thorsteinson suggests that you shouldn't be afraid to aim high...
- Trying to hire for creativity? You'll be interested in Madjar et al.'s research that indicated that depending on the type of creativity you're after (i.e., radical or incremental) both personal and environmental factors play a role. For example, if you're after radical creativity, look for a willingness to take risks and career commitment.
- You stats guys and gals out there will want to check out Johnson et al.'s study that investigated common method variance (CMV). Specifically, they found that by applying different types of remedies for CMV, they altered the relationship between two variables (core self-evaluation and job satisfaction). Something to consider when investigating criterion-related validity.
- Speaking of core self-evaluation, Chen disputes the inclusion of several factors in the concept, and argues that problems exist with its convergent and discriminant validity.
- Observer ratings of personality are hot. And Oh, Wang, and Mount get into the action with their study that found (as other have) that observer ratings yield higher predictive validities than self-reports. Now if we only had an easily-accessible database of observer ratings...
- Do hiring managers really discriminate against obese individuals? Yep, as Agerstrom and Rooth show. Specifically, scores on the implicit association test predicted whether hiring managers were likely to invite obese applicants for an interview.
- Looking to increase the number of staff that exhibit organizational citizenship behaviors? (ya know, things like helping out a co-worker when they don't have to) You might look first to the supervisor, as Yaffe and Kark demonstrate.
- A recent study that found that more women on a team increased group performance garnered some press, but the real story was the predictive role played by social competence (which women happened to score higher on). The authors pondered whether social competence could be trained, thus increasing group performance. Well, a new study by Kotsou et al. seems to suggest just that, showing that a group-format intervention increased emotional competence in adults (it also resulted in lower cortisol secretion and better subjective and physical well-being).
- Speaking of trying to maximize team performance, a study by Humphrey et al. found that both short- and long-term team performance was highest when variance in conscientiousness scores was lowest but variance in extroversion was highest. What does this mean? It suggests that people perform best when around others with similar levels of conscientiousness but perform better when working with people that vary in their levels of extroversion. Fascinating, and something to consider when hiring for or building teams.
- And speaking (again) of emotional competence, Seal et al. describe the development of a new measure of social and emotional development.
- Finally there is Hee et al. with an important study of prejudice. Specifically, the authors found that prejudice against out-group members increases with in-group size and perceptions of homogeneity among both the in-group and out-group. In addition to validating the importance of intergroup contact, this research suggests prejudice may be reduced when dealing with small groups and increasing an understanding of the differences within each group. Makes a lot of sense.
Last but not least, for you IPAC members don't forget that presentations from the just-concluded conference (which I heard was a rousing success) are starting to be posted in the members-only portion of the website. Good stuff!
- Looking to get a better salary offer? Research by Thorsteinson suggests that you shouldn't be afraid to aim high...
- Trying to hire for creativity? You'll be interested in Madjar et al.'s research that indicated that depending on the type of creativity you're after (i.e., radical or incremental) both personal and environmental factors play a role. For example, if you're after radical creativity, look for a willingness to take risks and career commitment.
- You stats guys and gals out there will want to check out Johnson et al.'s study that investigated common method variance (CMV). Specifically, they found that by applying different types of remedies for CMV, they altered the relationship between two variables (core self-evaluation and job satisfaction). Something to consider when investigating criterion-related validity.
- Speaking of core self-evaluation, Chen disputes the inclusion of several factors in the concept, and argues that problems exist with its convergent and discriminant validity.
- Observer ratings of personality are hot. And Oh, Wang, and Mount get into the action with their study that found (as other have) that observer ratings yield higher predictive validities than self-reports. Now if we only had an easily-accessible database of observer ratings...
- Do hiring managers really discriminate against obese individuals? Yep, as Agerstrom and Rooth show. Specifically, scores on the implicit association test predicted whether hiring managers were likely to invite obese applicants for an interview.
- Looking to increase the number of staff that exhibit organizational citizenship behaviors? (ya know, things like helping out a co-worker when they don't have to) You might look first to the supervisor, as Yaffe and Kark demonstrate.
- A recent study that found that more women on a team increased group performance garnered some press, but the real story was the predictive role played by social competence (which women happened to score higher on). The authors pondered whether social competence could be trained, thus increasing group performance. Well, a new study by Kotsou et al. seems to suggest just that, showing that a group-format intervention increased emotional competence in adults (it also resulted in lower cortisol secretion and better subjective and physical well-being).
- Speaking of trying to maximize team performance, a study by Humphrey et al. found that both short- and long-term team performance was highest when variance in conscientiousness scores was lowest but variance in extroversion was highest. What does this mean? It suggests that people perform best when around others with similar levels of conscientiousness but perform better when working with people that vary in their levels of extroversion. Fascinating, and something to consider when hiring for or building teams.
- And speaking (again) of emotional competence, Seal et al. describe the development of a new measure of social and emotional development.
- Finally there is Hee et al. with an important study of prejudice. Specifically, the authors found that prejudice against out-group members increases with in-group size and perceptions of homogeneity among both the in-group and out-group. In addition to validating the importance of intergroup contact, this research suggests prejudice may be reduced when dealing with small groups and increasing an understanding of the differences within each group. Makes a lot of sense.
Last but not least, for you IPAC members don't forget that presentations from the just-concluded conference (which I heard was a rousing success) are starting to be posted in the members-only portion of the website. Good stuff!
Sunday, June 12, 2011
Mega research update
Time for a big research update. There's a lot to catch up on, so I'll go quickly and give you just enough for you to follow up on if you're interested.
- More support for the attraction-selection-attrition model
- A 25-year review of the study of leadership outcomes
- Faces with stereotype-relevant features are more open to prejudice
- Early socioeconomic experience plays big role in future risk-taking behavior
- Taking the perspective of another may be key to reducing racial bias
- Generational differences in Big 5 personality factors
- How to improve management research
- Pros and cons of using social networking sites to make HR decisions
- Research directions for talent management
- Organizational branding and "best employer" surveys
- Using social networking sites for hiring decisions (if we just had a single database...)
- How computer adaptive testing aids in delivering unproctored internet tests
- How broad dimension factors may improve assessment centers
- Practical intelligence predicts success as an entrepreneur
- Psychological capital (efficacy, hope, optimism, resilience) is related to job performance
- Expressions of personality factors varies with the situation
Last but not least, this gem, a meta-analysis of the efficacy of simulation games for instruction. Relevance for us? As active engagement went up, so did learning. Lessons? Consider interactive simulations for recruitment and selection, but make sure the viewer is truly involved and not just a spectator.
- More support for the attraction-selection-attrition model
- A 25-year review of the study of leadership outcomes
- Faces with stereotype-relevant features are more open to prejudice
- Early socioeconomic experience plays big role in future risk-taking behavior
- Taking the perspective of another may be key to reducing racial bias
- Generational differences in Big 5 personality factors
- How to improve management research
- Pros and cons of using social networking sites to make HR decisions
- Research directions for talent management
- Organizational branding and "best employer" surveys
- Using social networking sites for hiring decisions (if we just had a single database...)
- How computer adaptive testing aids in delivering unproctored internet tests
- How broad dimension factors may improve assessment centers
- Practical intelligence predicts success as an entrepreneur
- Psychological capital (efficacy, hope, optimism, resilience) is related to job performance
- Expressions of personality factors varies with the situation
Last but not least, this gem, a meta-analysis of the efficacy of simulation games for instruction. Relevance for us? As active engagement went up, so did learning. Lessons? Consider interactive simulations for recruitment and selection, but make sure the viewer is truly involved and not just a spectator.
Sunday, April 17, 2011
Are executive recruiters discriminating against non-Whites?
You might think so if you just glanced at a recent study by Dreher, et al. in the Journal of Management. But what if we read the study...are the results more complicated?
Here's the background: The authors distributed a survey to over 13,000 U.S.-based BlueSteps subscribers. BlueSteps is an "executive career management service"--essentially a place where $100k+ job seekers and executive recruiters can connect, seemingly similar to TheLadders. It is maintained by the Association of Executive Search Consultants (AESC).
Due to various reasons (including the survey being blocked as spam), the researchers ended up with a final sample size of 572. So keep that in mind when interpreting the results. Also, nearly 90% of the respondents were male, and around the same number of respondents were White. So the non-white, non-male sample was relatively small.
What the researchers found was that white male respondents were significantly more likely to report being contacted by executive recruiters. And it wasn't because of factors like individual education or work experience--these were controlled for. Interestingly, further analysis revealed that race appeared to be the driving force; there was no difference in contact between female and non-White male respondents.
They also found that switching employers ("pursuing an external labor market strategy") resulted in a compensation premium only for White male respondents. Not surprisingly, those who reported receiving more contacts from search firms had greater compensation. But White males who had received the most number of contacts reported the highest level of compensation compared to other demographic groups.
Here's how the authors summarized their results: "...our study suggests that the White male advantage associated with external job change is, to a meaningful degree, sensitive to the processes and practices of executive search firms."
The authors do point out that the small size of the non-White male sample may have something to do with the results, and they caution readers about generalizing the results to other populations. However, they also state: "...the executive search industry would likely benefit from in-depth internal reviews regarding a variety of diversity-oriented issues. It is in the search firm’s likely best interest to present a diverse slate of candidates to its clients; as such, search firms would benefit from pursuing internal studies designed to determine if and why female and minority male managers and executives are underrepresented in their databases (or in databases of the BlueSteps variety)."
Is this an example of blatant racial discrimination? It's suggestive but not conclusive. The authors even write "we are not suggesting that anything sinister is going on or that search firms are intentionally discriminating against women and non-White males." But the fact that a large percentage of top search professionals appear to be White males should factor into the conversation. It may not be intentional, but that doesn't mean it's not discrimination. More research (e.g., controlled laboratory studies) would help us answer this question.
Here's the background: The authors distributed a survey to over 13,000 U.S.-based BlueSteps subscribers. BlueSteps is an "executive career management service"--essentially a place where $100k+ job seekers and executive recruiters can connect, seemingly similar to TheLadders. It is maintained by the Association of Executive Search Consultants (AESC).
Due to various reasons (including the survey being blocked as spam), the researchers ended up with a final sample size of 572. So keep that in mind when interpreting the results. Also, nearly 90% of the respondents were male, and around the same number of respondents were White. So the non-white, non-male sample was relatively small.
What the researchers found was that white male respondents were significantly more likely to report being contacted by executive recruiters. And it wasn't because of factors like individual education or work experience--these were controlled for. Interestingly, further analysis revealed that race appeared to be the driving force; there was no difference in contact between female and non-White male respondents.
They also found that switching employers ("pursuing an external labor market strategy") resulted in a compensation premium only for White male respondents. Not surprisingly, those who reported receiving more contacts from search firms had greater compensation. But White males who had received the most number of contacts reported the highest level of compensation compared to other demographic groups.
Here's how the authors summarized their results: "...our study suggests that the White male advantage associated with external job change is, to a meaningful degree, sensitive to the processes and practices of executive search firms."
The authors do point out that the small size of the non-White male sample may have something to do with the results, and they caution readers about generalizing the results to other populations. However, they also state: "...the executive search industry would likely benefit from in-depth internal reviews regarding a variety of diversity-oriented issues. It is in the search firm’s likely best interest to present a diverse slate of candidates to its clients; as such, search firms would benefit from pursuing internal studies designed to determine if and why female and minority male managers and executives are underrepresented in their databases (or in databases of the BlueSteps variety)."
Is this an example of blatant racial discrimination? It's suggestive but not conclusive. The authors even write "we are not suggesting that anything sinister is going on or that search firms are intentionally discriminating against women and non-White males." But the fact that a large percentage of top search professionals appear to be White males should factor into the conversation. It may not be intentional, but that doesn't mean it's not discrimination. More research (e.g., controlled laboratory studies) would help us answer this question.
Saturday, November 20, 2010
New issues of Personnel Psych and IJSA: Muslim applicants, selection perceptions, and more
Two of the big journals have come out with new issues, so let's take a look at the highlights:
First, in the Winter 2010 issue of Personnel Psychology:
King and Ahmad describe the results of several experiments that found interviewers and raters altered their behavior for confederates exhibiting obvious Muslim-identified behavior (e.g., clothing) depending on whether applicants exhibited stereotype-inconsistent behavior. For those that didn't, reactions were shorter and more negative. On the other hand, no difference was found in offers between those dressed in Muslim-identified clothing and those that weren't. So behavior--specifically its stereotypicality--and not simply something obvious like dress, may be key in predicting/preventing discriminatory behavior.
Do you consistently read the "Limitations" section of journal articles? Brutus et al. did, for three major I/O journals from 1995 to 2008 and found that threats to internal validity were the most commonly reported limitation. Interestingly, they also found that the nature of limitations reported changed over time (e.g., more sampling issues due to volunteers, variance issues). You can see an in press version here.
Next up, Henderson reports impressive criterion-related validity (combined operational = .86) for a test battery consisting of a g-saturated exam and a strength/endurance exam after following a firefighter academy class for 23 years. He suggests that employers have considerable latitude in choosing exams as long as they are highly loaded on these two factors, and also suggests approximately equal weighting.
Struggling to communicate the utility of sound assessment? Last but not least, Winkler et al. describe the results of sharing utility information with a sample of managers and found that using a casual chain analysis--rather than simply a single attribute--increased understanding, perceived usefulness, and intent to use.
Let's switch now to the December 2010 issue of the International Journal of Selection and Assessment (IJSA). There's a lot of great content in this issue, so take a deep breath:
Interested in unproctored internet testing? You'll want to check out Guo and Drasgow's piece on verification testing. The authors recommend using a Z-test over the likelihood ratio test for detecting cheating, although both did very well.
Walsh et al. discuss the moderating effect that cultural practices (performance orientation, uncertainty avoidance) have on selection fairness perceptions.
Speaking of selection perceptions, Oostrom et al. found that individual differences play a role in determining how applicants respond--particularly openness to experience. The authors recommend considering the nature of your applicant pool before implementing programs to improve perceptions of the assessment process.
Those of you interested in cut scores (and hey, who isn't) should check out Hoffman et al.'s piece on using a difficulty-anchored rating scale and the impact it has on SME judgments.
Back to perceptions for a second, Furnham and Chamorro-Premuzic asked a sample of students to rate seventeen different assessment methods for their accuracy and fairness. Not surprisingly, panel interviews and references came out on top in terms of fairness, while those that looked the most like a traditional test (e.g., drug, job knowledge, intelligence) were judged least accurate and fair. Interestingly, self-assessed intelligence moderated the perceptions (hey, if I think I'm smart I might not mind intelligence tests!).
And now for something completely different (those of you that get that reference click here for a trip down memory lane). A study by Garcia-Izquierdo et al. of information contained in online job application forms from a sample of companies found on the Spanish Stock Exchange. A surprisingly high percentage of firms asked for information on their applications that at best would be off-putting, at worst could lead to lawsuits, such as age/DOB, nationality, and marital status. The authors suggest this area of e-recruitment is ripe for scientist-practitioner collaboration.
Last but not least, a piece that ties the major topics of this post together: selection perceptions and recruitment. Schreurs et al. gathered data from 340 entry-level applicants to a large financial services firm and found that applicant perceptions, particularly of warmth/respect, mediated the relationship between expectations and attraction/pursuit intentions. This reinforces other research that has underlined the importance of making sure organizational representatives put your best foot/face forward.
First, in the Winter 2010 issue of Personnel Psychology:
King and Ahmad describe the results of several experiments that found interviewers and raters altered their behavior for confederates exhibiting obvious Muslim-identified behavior (e.g., clothing) depending on whether applicants exhibited stereotype-inconsistent behavior. For those that didn't, reactions were shorter and more negative. On the other hand, no difference was found in offers between those dressed in Muslim-identified clothing and those that weren't. So behavior--specifically its stereotypicality--and not simply something obvious like dress, may be key in predicting/preventing discriminatory behavior.
Do you consistently read the "Limitations" section of journal articles? Brutus et al. did, for three major I/O journals from 1995 to 2008 and found that threats to internal validity were the most commonly reported limitation. Interestingly, they also found that the nature of limitations reported changed over time (e.g., more sampling issues due to volunteers, variance issues). You can see an in press version here.
Next up, Henderson reports impressive criterion-related validity (combined operational = .86) for a test battery consisting of a g-saturated exam and a strength/endurance exam after following a firefighter academy class for 23 years. He suggests that employers have considerable latitude in choosing exams as long as they are highly loaded on these two factors, and also suggests approximately equal weighting.
Struggling to communicate the utility of sound assessment? Last but not least, Winkler et al. describe the results of sharing utility information with a sample of managers and found that using a casual chain analysis--rather than simply a single attribute--increased understanding, perceived usefulness, and intent to use.
Let's switch now to the December 2010 issue of the International Journal of Selection and Assessment (IJSA). There's a lot of great content in this issue, so take a deep breath:
Interested in unproctored internet testing? You'll want to check out Guo and Drasgow's piece on verification testing. The authors recommend using a Z-test over the likelihood ratio test for detecting cheating, although both did very well.
Walsh et al. discuss the moderating effect that cultural practices (performance orientation, uncertainty avoidance) have on selection fairness perceptions.
Speaking of selection perceptions, Oostrom et al. found that individual differences play a role in determining how applicants respond--particularly openness to experience. The authors recommend considering the nature of your applicant pool before implementing programs to improve perceptions of the assessment process.
Those of you interested in cut scores (and hey, who isn't) should check out Hoffman et al.'s piece on using a difficulty-anchored rating scale and the impact it has on SME judgments.
Back to perceptions for a second, Furnham and Chamorro-Premuzic asked a sample of students to rate seventeen different assessment methods for their accuracy and fairness. Not surprisingly, panel interviews and references came out on top in terms of fairness, while those that looked the most like a traditional test (e.g., drug, job knowledge, intelligence) were judged least accurate and fair. Interestingly, self-assessed intelligence moderated the perceptions (hey, if I think I'm smart I might not mind intelligence tests!).
And now for something completely different (those of you that get that reference click here for a trip down memory lane). A study by Garcia-Izquierdo et al. of information contained in online job application forms from a sample of companies found on the Spanish Stock Exchange. A surprisingly high percentage of firms asked for information on their applications that at best would be off-putting, at worst could lead to lawsuits, such as age/DOB, nationality, and marital status. The authors suggest this area of e-recruitment is ripe for scientist-practitioner collaboration.
Last but not least, a piece that ties the major topics of this post together: selection perceptions and recruitment. Schreurs et al. gathered data from 340 entry-level applicants to a large financial services firm and found that applicant perceptions, particularly of warmth/respect, mediated the relationship between expectations and attraction/pursuit intentions. This reinforces other research that has underlined the importance of making sure organizational representatives put your best foot/face forward.
Saturday, June 26, 2010
June '10 IOP Part II: "Test bias"
In Part I of this two-part post, I described the first focal article in the June 2010 issue of Industrial and Organizational Psychology (IOP), devoted to emotional intelligence. In this post, I will describe the second focal article, which focuses on test bias.The focal article is written by Meade & Tonidandel and has at its core one primary argument. In their words:
"...the most commonly used technique for evaluating whether a test is biased, the regression-based approach outlined by Cleary (1968), is itself flawed and fails to tell us what we really want to know."
The Cleary method of detecting test bias is conducted by regressing a criterion (e.g., job performance) on test scores along with a dichotomous demographic grouping variable, such as sex or race, and looking at the interaction. If there is a significant difference by group variable (e.g., different slope, different intercepts), this suggests the test may be "biased". This situation is called differential prediction or predictive bias. The authors contrast this with "internal methods" of examining tests for bias, such as differential item and test functioning and confirmatory factor analysis which do not rely upon any external criterion data.
Meade & Tonidandel state that while the Cleary method has become the de facto method for evaluating bias, there are several significant flaws with the approach. Most critically, the presence of differential prediction does not necessarily mean the test itself is the cause (which we would call measurement bias). Other potential causes are:
- bias in the criterion
- reliability of the test
- omitted variables (i.e., important predictors are left out of the regression)
Another important limitation of the Cleary method is the susceptibility of slope difference tests to low power--this can result in findings of no slope differences due to small samples rather than a true absence. In addition, because of the type I error rate of the intercept test, one is likely to conclude that intercept differences are present when none truly exist.
Because of these limitations, the authors recommend the following general steps when investigating a test:
1. Conduct internal analyses examining differential functioning of items and the test.
2. Examine both test and criterion scores for significant group mean differences before conducting regression analyses.
3. Compute d effect size estimates for group mean differences for both the test and the criterion.
The authors present several scenarios where tests may be "biased" as conceived in the traditional sense but may or may not be "fair"--an important distinction. For example, one group may have higher performance scores, but there is no group difference in test scores. Use of the predictor may result in one group being hired at greater or lesser rates than they "should be", but our judgment of the fairness of this requires consideration of organizational and societal goals (e.g., affirmative action, maximum efficiency) rather than simply an analysis of the tests.
The authors close with several overall recommendations:
1) Stop using the term test bias (too many interpretations, confounds different concepts).
2) Always examine both measurement bias and differential prediction.
3) Do not assume a test is unusable if differential prediction is indicated.
There are several commentary articles that follow the focal one. The authors of these pieces make several points, ranging from criticisms of techniques and specific statements to suggestions for further analyzing the issue. Some of the better comments/questions include:
- How likely is it that the average practitioner is aware of these issues and further is able to conduct these analyses? (a point the authors agree with in part)
- This approaches advocated mainly work with multi-item tests; things get more complicated when we're using several tests or an overall rating.
- It may not be helpful to create a "recipe" of recommendations (as listed above); rather we should acknowledge that each selection scenario is a different context.
- We are still ignoring the (probably more important) issue of why a test may or may be "biased". An important consideration is the nature of group membership in demographic categories (including multiple group memberships).
Meade & Tonidandel provide a response to the commentaries and acknowledge several valid points raised but end this article with the same proposition that they started the focal article with:
"For 30 years, the Cleary model has served as the dominant guidelines for analyses of test bias in our field. We believe that these should be revisited."
Thursday, May 27, 2010
Lewis case emphasizes need for valid tests
On Monday, May 24, the U.S. Supreme Court ruled in Lewis v. City of Chicago that plaintiffs filing an adverse impact discrimination claim under Title VII of the Civil Rights Act have 300 days to file from each time the test is used to fill a position--not just 300 days from when the test was administered.
For the most part, this mainly impacts employers that run an exam and use the results to fill positions for several years. In this case it was a firefighter entry-level exam, and my guess is it will mostly be public sector agencies and large employers that should pay particular attention to the ruling.
Why? Well for one it means more potential adverse impact lawsuits. If you were counting on being safe after 300 days from the exam, that is no longer the case. Second, it emphasizes the need to follow professional guidelines when developing an exam. Employers can successfully defend against an adverse impact case by showing that the selection practice is "job related for the position in question and consistent with business necessity..." (and that no alternatives with similar validity with less adverse impact were available). This means your exams need to be developed and interpreted by people who know what they're doing.
The City claimed that evidence related to an employer's business necessity defense might be unavailable by the time the lawsuit is brought--the court was not swayed. This means you'll want to hang on to your exam development records for at least a year beyond the last time you use the results to fill a position.
Another point worth noting: this case boils down to the validity of a cut score used by the City, which they themselves admitted wasn't supportable. Proper exam development and interpretation includes setting a job-related pass point based on subject matter input and/or statistical evidence that it is linked to job performance.
The Lewis ruling doesn't fundamentally change what we should be doing. It just emphasizes that we need to do it right.
Click here for a quick overview of the facts of the case.
Friday, May 21, 2010
IPAC conference to feature Campbell, McDaniel, Highhouse, and more
Those of you on the fence about attending the 2010 International Personnel Assessment Council (IPAC) conference on July 18-21 may be interested to know that a preliminary schedule has been released that reveals some great speakers and topics. For example:
- David Campbell's provocatively titled opening session, The Use of Picture Postcards for Exploring Diversity Issues Such as Bias and Prejudices, or "How Can We Keep Our Grandchildren From Going to War With Each Other?"
- Not to be outdone, Michael McDaniel kicks things off Tuesday morning with Abolish the Uniform Guidelines.
- Scott Highhouse closes things up Wednesday with A Critical Look at Holistic Assessment
- Great pre-conference workshops on everything from job analysis to fairness
- Wonderfully diverse concurrent sessions on topics such as public service motivation, leadership coaching, simulations, engagement, online testing, charging for exams, test transportability, cross-cultural personality assessment, measuring workforce gaps, adverse impact analysis, faking and lie detection, and succession planning. And that's just a sample!
Staying current on assessment through professional education is one of the commandments of our field. I hope you'll be joining your friends and colleagues in Newport Beach. Early bird registration ends June 1st.
- David Campbell's provocatively titled opening session, The Use of Picture Postcards for Exploring Diversity Issues Such as Bias and Prejudices, or "How Can We Keep Our Grandchildren From Going to War With Each Other?"
- Not to be outdone, Michael McDaniel kicks things off Tuesday morning with Abolish the Uniform Guidelines.
- Scott Highhouse closes things up Wednesday with A Critical Look at Holistic Assessment
- Great pre-conference workshops on everything from job analysis to fairness
- Wonderfully diverse concurrent sessions on topics such as public service motivation, leadership coaching, simulations, engagement, online testing, charging for exams, test transportability, cross-cultural personality assessment, measuring workforce gaps, adverse impact analysis, faking and lie detection, and succession planning. And that's just a sample!
Staying current on assessment through professional education is one of the commandments of our field. I hope you'll be joining your friends and colleagues in Newport Beach. Early bird registration ends June 1st.
Saturday, April 10, 2010
April 2010 TIP and latest EEO Insight
The April 2010 issue of The Industrial/Organizational Psychologist (TIP) is now available here. Let's check out some of what's inside:
- Several great, touching dedications to the late great Frank Landy. I found Rick Jacobs' eulogy particularly moving. My first real job out of graduate school was working for Frank doing research for expert witness testimony. Fabulous experience, amazing individual.
- Joel Wiesen proposes a novel approach to promotional selection in fire department settings.
- A great little article about moving from an I/O position to an HR generalist one. Very timely given all the changes HR departments are experiencing.
- The current status of legal protection for sexual minorities in the workplace
- A great article about the Bridgeport, CT case and why both the media and the city got things wrong.
- A fascinating breakdown of the major activities of I/O consultants v. internal practitioners v. academics. Check out Table 3.
In addition, there's a new issue of EEO Insight with some great content, and one article in particular I'd like to point out: A comprehensive look at how to successfully develop diversity initiatives and testing programs post-Ricci, including the essential elements of a Croson study (see Table 2) and the "strong basis in evidence" standard as applied to several practical examples (Table 3). It's a keeper and starts on page 27.
Some other articles to check out include:
- Strategies for defending and framing the issue of adverse impact in selection
- How to avoid adverse impact when choosing a test
- Several great, touching dedications to the late great Frank Landy. I found Rick Jacobs' eulogy particularly moving. My first real job out of graduate school was working for Frank doing research for expert witness testimony. Fabulous experience, amazing individual.
- Joel Wiesen proposes a novel approach to promotional selection in fire department settings.
- A great little article about moving from an I/O position to an HR generalist one. Very timely given all the changes HR departments are experiencing.
- The current status of legal protection for sexual minorities in the workplace
- A great article about the Bridgeport, CT case and why both the media and the city got things wrong.
- A fascinating breakdown of the major activities of I/O consultants v. internal practitioners v. academics. Check out Table 3.
In addition, there's a new issue of EEO Insight with some great content, and one article in particular I'd like to point out: A comprehensive look at how to successfully develop diversity initiatives and testing programs post-Ricci, including the essential elements of a Croson study (see Table 2) and the "strong basis in evidence" standard as applied to several practical examples (Table 3). It's a keeper and starts on page 27.
Some other articles to check out include:
- Strategies for defending and framing the issue of adverse impact in selection
- How to avoid adverse impact when choosing a test
Thursday, February 04, 2010
Lessons from NYC Fire case - part 2
Part 2 of 2
Last time I discussed five important lessons we can take away from recent rulings in the Vulcan v. City of New York case. In this post I'll review the remaining lessons and also discuss the relief order.
----
6) The city failed to provide sufficient evidence that the exam(s) tested for a sufficient number of the critical KSAs. They also failed to explain why they chose not to measure several KSAs identified as critical.
Lesson: the courts do not require employers to measure every single critical KSA. But there is an expectation that employers attempt to measure a sufficient number that represent a significant portion of the job requirements. In this case, that included non-cognitive abilities such as resistance to stress, teamwork, and conscientiousness, that were not measured.
7) The city failed to adequately consider how to measure a significant number of essential KSAs. While some of their concerns were valid (e.g., structured interviews for all applicants would be an operational nightmare), there are many different forms of testing that should have been considered, including situational judgment tests (SJTs) and biodata, which can be used to measure non-cognitive components.
Lesson: triers of fact expect employers to be up on the various assessment methods available and be able to explain why they chose not to use certain ones. This includes tests that are relatively easy to develop (e.g., SJTs) as well as ones that require substantial resources and statistical expertise (e.g., biodata).
8) The city failed to conduct a reading level analysis on the exams to ensure that it was not "pointlessly high." The plaintiff introduced evidence suggesting the reading level was above 12th grade; in addition, it appeared to exceed the reading level of materials at the academy.
Lesson: never forget that every assessment method is in some sense measuring additional KSAs beyond those you intend. For written exams, reading comprehension is always a requirement (barring accommodation). It's quite easy to conduct a reading level analysis (MS Word has it built in) to ensure that the level is reasonable and matches other job-related material.
9) The city failed to show that the cutoff scores (pass points) established for the exams were based on adequate rationale, namely "the necessary qualifications for the job of entry-level firefighter." Instead, the cutoff scores were based on operational need (the number of job openings expected). This is particularly important in multiple-hurdle selection processes such as in this case, where a failure on one exam component precludes an applicant from participating in the rest of the (potentially compensatory) assessment process.
Lesson: ultimately applicants have to pass the test(s) to be considered for employment. Cutoff scores should be established using the expertise of both SMEs and test developers and should be based on the minimum competency levels required upon entry to the job. At a minimum (and I would not rely solely upon this), the scores should be analyzed to identify any logical "break-points."
----
After ruling for the plaintiffs on both the adverse impact and disparate treatment claims, the judge issued a relief order on 1/10/10. In it, he imposes several things, including the following:
1) The city must develop a new testing procedure for entry-level firefighter in conjunction with the relevant parties. Following the development of the test, there will be a hearing to determine if this test should be used rather than the current test (developed in 2007 and not at issue in this litigation).
2) The court shall develop a process by which the approximately 7,400 applicants covered by this case can file a claim for monetary relief.
3) The city will identify 293 black candidates on the eligibility list and offer them priority hiring. (No quotas are being imposed, although the judge leaves this possibility open)
4) Retroactive seniority for those hired.
In addition, several other issues are up for debate, including the appointment of a special master or monitor, standards that will be relied upon in constructing the new exam, and the need for additional relief.
---
So what did we learn from all this? If you follow--fairly closely--best practices when developing and administering exams, you will be on solid ground defending them. If you don't, and your exam has a discriminatory effect, you may be called on it--and it's not a pleasant process. I'll leave you with this quote from the January ruling on disparate treatment:
"The history of the City's efforts to remedy its discriminatory firefighter hiring policies can be summarized as follows: 34 years of intransigence and deliberate indifference, bookeneded by identical judicial declarations that the City's hiring policies are illegal."
Last time I discussed five important lessons we can take away from recent rulings in the Vulcan v. City of New York case. In this post I'll review the remaining lessons and also discuss the relief order.
----
6) The city failed to provide sufficient evidence that the exam(s) tested for a sufficient number of the critical KSAs. They also failed to explain why they chose not to measure several KSAs identified as critical.
Lesson: the courts do not require employers to measure every single critical KSA. But there is an expectation that employers attempt to measure a sufficient number that represent a significant portion of the job requirements. In this case, that included non-cognitive abilities such as resistance to stress, teamwork, and conscientiousness, that were not measured.
7) The city failed to adequately consider how to measure a significant number of essential KSAs. While some of their concerns were valid (e.g., structured interviews for all applicants would be an operational nightmare), there are many different forms of testing that should have been considered, including situational judgment tests (SJTs) and biodata, which can be used to measure non-cognitive components.
Lesson: triers of fact expect employers to be up on the various assessment methods available and be able to explain why they chose not to use certain ones. This includes tests that are relatively easy to develop (e.g., SJTs) as well as ones that require substantial resources and statistical expertise (e.g., biodata).
8) The city failed to conduct a reading level analysis on the exams to ensure that it was not "pointlessly high." The plaintiff introduced evidence suggesting the reading level was above 12th grade; in addition, it appeared to exceed the reading level of materials at the academy.
Lesson: never forget that every assessment method is in some sense measuring additional KSAs beyond those you intend. For written exams, reading comprehension is always a requirement (barring accommodation). It's quite easy to conduct a reading level analysis (MS Word has it built in) to ensure that the level is reasonable and matches other job-related material.
9) The city failed to show that the cutoff scores (pass points) established for the exams were based on adequate rationale, namely "the necessary qualifications for the job of entry-level firefighter." Instead, the cutoff scores were based on operational need (the number of job openings expected). This is particularly important in multiple-hurdle selection processes such as in this case, where a failure on one exam component precludes an applicant from participating in the rest of the (potentially compensatory) assessment process.
Lesson: ultimately applicants have to pass the test(s) to be considered for employment. Cutoff scores should be established using the expertise of both SMEs and test developers and should be based on the minimum competency levels required upon entry to the job. At a minimum (and I would not rely solely upon this), the scores should be analyzed to identify any logical "break-points."
----
After ruling for the plaintiffs on both the adverse impact and disparate treatment claims, the judge issued a relief order on 1/10/10. In it, he imposes several things, including the following:
1) The city must develop a new testing procedure for entry-level firefighter in conjunction with the relevant parties. Following the development of the test, there will be a hearing to determine if this test should be used rather than the current test (developed in 2007 and not at issue in this litigation).
2) The court shall develop a process by which the approximately 7,400 applicants covered by this case can file a claim for monetary relief.
3) The city will identify 293 black candidates on the eligibility list and offer them priority hiring. (No quotas are being imposed, although the judge leaves this possibility open)
4) Retroactive seniority for those hired.
In addition, several other issues are up for debate, including the appointment of a special master or monitor, standards that will be relied upon in constructing the new exam, and the need for additional relief.
---
So what did we learn from all this? If you follow--fairly closely--best practices when developing and administering exams, you will be on solid ground defending them. If you don't, and your exam has a discriminatory effect, you may be called on it--and it's not a pleasant process. I'll leave you with this quote from the January ruling on disparate treatment:
"The history of the City's efforts to remedy its discriminatory firefighter hiring policies can be summarized as follows: 34 years of intransigence and deliberate indifference, bookeneded by identical judicial declarations that the City's hiring policies are illegal."
Wednesday, December 30, 2009
Outback settlement contains interesting requirements
You may have heard that Outback Steakhouse, a restaurant chain based in Tampa, Florida, has agreed to settle a gender discrimination lawsuit for $19M. What's interesting about this isn't the size of the settlement, but rather the conditions attached.
Background: The EEOC sued Outback in 2006, claiming it systematically discriminated against its female employees by denying them promotion opportunities to the more lucrative profit-sharing management positions. In addition, they claimed that female employees were denied promotional job assignments such as kitchen management, which were required for employees to be considered for top management positions.
The settlement: Outback agreed to a four-year consent decree and $19M in monetary relief. So far, pretty standard. But there were additional settlement requirements, and here's where it gets interesting. In addition to the monetary relief, Outback has agreed to:
1. Create an online application system for employees interested in management positions. This is the first time I've seen this in a settlement (which isn't to say it hasn't happened) and seems to indicate that the EEOC views this as a more "objective" screening mechanism.
2. Create and hire someone for a newly created "human resources executive" position titled Vice President of People. Again, this is a new one for me.
3. Hire an outside consultant for at least two years who will monitor the online application system to ensure women are being provided equal opportunities for promotion and provide reports to the EEOC every 6 months.
The main thing that strikes me about this settlement is the faith that is being placed in an online application system to somehow ensure equal opportunity. Sure, having a standardized application system may cut down on some of the subjectivity of individual hiring supervisors, but it leaves me wondering:
- What will the screening criteria for management positions be?
- How will the outside consultant define "equal opportunities"?
- How will access to the online system be controlled, and who will be making screening/hiring decisions?
- What happens if there continues to be adverse impact, which you would expect if applicants continue to be screened on experience?
- What will be the duties of the Vice President of People, how will they be hired, and how will they interact with the consultant?
This will be interesting to watch.
Monday, October 19, 2009
Is recruiting using SNS discriminatory?
I keep reading/hearing about how recruiting using social networking sites (SNS) opens employers up to discrimination lawsuits because of who uses the sites. For the most part, this just plain isn't true.A recent Pew study is the latest to show that when it comes to using SNS like Facebook, MySpace, and LinkedIn, you really should have one primary demographic concern when it comes to ensuring a diverse candidate pool: age.
Not gender, at least not in traditional sense. While four years ago SNS users tilted slightly male (55%), the balance has essentially flipped today (54% female).
Not race, there simply do not appear to be generalizable differences in racial groups when it comes to these sites (in fact I've seen some data that suggest the user base on these sites is more diverse)--but things change, and this may vary with particular sites, so keep an eye on this one.
But when it comes to age, SNS users are disproportionately younger than the overall Internet population. In the words of the Pew report, "[this] doesn't mean that more older adults aren't flocking to SNS--they are--but younger adults are ALSO flocking to the sites, so the overall representation of the age cohorts in the SNS user population has actually gotten younger."
One demographic difference I don't see a whole lot about: disability status. Are individuals with disabilities more/less likely to use SNS? I think that's an important question we need to address if we're truly trying to diversity our candidate pools.
Subscribe to:
Posts (Atom)