Research update

I can't believe it's been three months since a research update.  I was waiting until I got critical mass, and with the release of the September issues of IJSA, I think I've hit it.

So let's start there:

- Experimenting with using different rating scales on SJTs (with "best and worst" response format doing the best of the traditional scales)

- Aspects of a semi-structured interview added incremental validity over cognitive ability in predicting training performance

- Studying the use of preselection methods (e.g., work experience) prior to assessment centers in German companies

- The proposed general factor of personality may be useful in selection contexts (this one was a military setting)

- Evidence that effective leaders show creativity and political skill

- Investigating the relationship (using survey data) between personality facets and CWBs (with emotional stability playing a key role)

- Corrections for indirect range restriction boosted the upper end of structured interview validity substantially

- A method of increasing the precision of simulations that analyze group mean differences and adverse impact

- A very useful study that looked at the prediction of voluntary turnover as well as performance using biodata and other applicant information, including recruitment source, among a sample of call center applicants.  Reuslts?  Individuals who had previously applied, chose to submit additional information, were employed, or were referrals had significantly less voluntary turnover.



Moving on...let's check out the May issue of JAP; there are only two articles but both worth looking at:

- First, a fascinating study of the firm-level impact of effective staffing and training, suggesting that the former allow organizations greater flexibility and adaptability (e.g., to changing financial conditions).

- Second, another study of SJT response formats.  The researchers found, using a very large sample, the "rate" format (e.g., "rate each of the following options in terms of effectiveness") to be superior in terms of validity, reliability, and group differences.


Next, the July issue of JOB, which is devoted to leadership:

- You might want to check out this overview/critique of the various leadership theories.

- This study suggests that newer models proposing morality as an important component of leadership success have methodological flaws.

- Last, a study of why Whites oppose affirmative action programs

Let's move to the September issue of Industrial and Organizational Psychology:

- The first focal article discusses the increasing movement of I/O psychology to business schools.  The authors found evidence that this is due in large part to some of the most active and influential I/O researchers moving to business schools.

- The second is about stereotype threat--specifically its importance as a psychological construct and the paucity of applied research about it.


Coming into the home stretch, the Summer issue of Personnel Psych:

- The distribution of individual performance may not be normal if, as these researchers suggest, "star performers" have emerged

- Executives with high levels of conscientiousness and who display transformational leadership behavior may directly contribute to organizational performance

Rounding out my review, check out a few recent articles from PARE:

- I'm not even gonna attempt to summarize this, so here's the title: Multiple-Group confirmatory factor analysis in R – A tutorial in measurement invariance with continuous and ordinal indicators

- Improving exploratory factor analysis for ordinal data

- Improving multidimensional adaptive testing


Last but not least, it's not related to recruitment or assessment, but check out this study that found productivity increases during bad weather :)

That's all folks!

Another (smaller) research update

Seems like this always happens, I post a research update and then Personnel Psychology comes out with their latest issue.

Oh well!  At least you won't have to read as much this time  And while we're at it, let's take a look at the latest Journal of Applied Social Psychology.  But first, Personnel Psych:

- The relationship between task performance and citizenship behavior: more complicated than you might think

- Propensity scoring: a statistical technique that may improve our ability to make causal inferences from quasi-experimental designs

- Can creative employees drive higher organizational performance?  Well that seems to depend on how open to risk your organization is...

- It's easier said than done, but those looking for a job should try to stay positive: it influences how quickly you're likely to land your next position

- Last but not least, a small correction to a study posted earlier on the advantage of contextualizing personality inventory items


Okay, on to June 2013 JASP, which is a special issue on prejudice:

-Trying to reduce bias in your selection process? Make sure your raters get enough sleep.

- Discrimination against people based on their weight: it starts early!  (there's another study on weight discrimination that looked at its impact on perceived social status)

- Discrimination against men perceived to be Muslim increased among Western observers when subjects were presented in traditional dress.

- Those of Black/White biracial descent were less likely to be perceived as minorities and thus less appropriate receipts of affirmative action

- Okay, I like this one in particular, and it happens to be the last one I'll describe.  Researchers showed that competency perceptions of an African-American man hired varied depending on whether it was under "affirmative action" or a "diversity initiative"--the latter resulting in higher ratings.  This was particularly true for conservative, White raters.  Just goes to show how important words are, as any political consultant will tell you.

Fin.

June 2010 IJSA

The summer journal season continues with the June 2010 issue of the International Journal of Selection and Assessment. Take a deep breath, there's a lot of stuff packed into this issue:

- Roth et al. provide evidence that women outperformed men on work sample exams that involved social skills, writing skills, or a broad array of KSAs. To the extent that an employer is trying to avoid discriminating against female applicants, this provides support for work sample usage.

- In a study of managers in Taiwan, Tsai et al. show that the most effective way an applicant can make up for a slip in an interview is to apologize (vs. attempting to justify or use an excuse).

- Jackson et al. strive to add some clarity on task-based assessment centers

- Blickle & Schnitzler provide evidence of the construct and criterion-related validity of the political skill inventory

- Colarelli et al. studied how racial prototypicality and affirmative action policies impact hiring decisions. Results of a resume review indicated more jobs were awarded to black candidates as racial prototypicality and affirmative action policy strength increased, but stronger AA policies decreased the percentage of minority hires attributed to higher qualifications.

- In my personal favorite article of the issue, Karl et al. found in a study of U.S. and German students that those low on conscientiousness (especially), agreeableness, and emotional stability were more likely to post "Facebook Faux Pas". This provides some support for employers who screen out applicants based on inappropriate social networking posts. I'll talk more about this in my upcoming webinar.

- Denis, et al. provide support for the NEO PI-R's ability to predict job performance in two French-Canadian samples.

- Bilgiç and Acarlar report results of a study of Turkish students and perceptions of various selection instruments. Interviews were rated most highly and there were some differences in terms of privacy perceptions depending on the goal orientation of the student.

- Trying to figure out how to hire better direct support professionals (e.g., those providing long-term residential care or care to those with disabilities)? Robson, et al. describe the development of a composite predictor composed of various measures (e.g., agreeableness, numerical ability) that predicted performance, satisfaction, and turnover.

- Ahmetoglu et al. provide support for using the Fundamental Interpersonal Relationship Orientations-Behaviour (FIRO-B) to predict leadership capability.

- Ispas et al. describe results of a study that showed support for a nonverbal cognitive ability measure (the GAMA) in predicting job performance in two samples.

- Last but not least, in another win for context-specific assessments, Pace & Brannick show how a measure of openness to experience tailored to specific work outpredicted the comparable general NEO PI-R scale. IMHO this is how personality measures will eventually become more prominent and accepted as pre-hire assessments.

Latest EEO Insight

EEO Insight is quickly becoming a great resource for anyone interested in issues related broadly to equal employment opportunity. And this isn't just affirmative action plans--it includes anyone interested in recruitment and assessment.

In the latest issue (v1, #3), you'll read about:

- Alternatives to RIFs such as wage freezes and job sharing and the EEO implications

- Analyzing layoff decisions for statistical evidence of adverse impact

- Using multiple regression to detect race and gender differences in compensation

- Ricci in retrospect and lessons learned

- Reaching out to veterans and individuals with disabilities

- Results of the EEO best practices survey and (very good) recommendations

By the way, if you're interested in EEO issues and you're not already reading OFCCP Blog Spot, I highly recommend starting.

Ricci case: Full of sound and fury...

There's been a lot of hoopla over the last several days over the U.S. Supreme Court's decision in Ricci v. DeStefano. It's been described as a win for "reverse discrimination" cases, a rebuke of written tests, and judicial activism. The way I read it, the decision is completely unsurprising and will likely change absolutely nothing about employment testing.

For anyone who isn't familiar with the case, here's a very brief rundown: the City of New Haven, CT gave promotional tests for Lieutenant and Captain firefighter positions using written multiple choice tests and interviews. When they crunched the results it turned out--not surprisingly--that there was statistical evidence of adverse impact against the Black candidates. The City decided not to use the list, and the White and Hispanic candidates sued, claiming disparate treatment. The Supreme Court ruled in their favor.

A little unusual of a case in terms of who's on what side, and there's a lot of good reading in the decision for anyone wanting to know more about test validation. But the decision itself is totally consistent with three main themes from previous decisions:

(1) There really isn't "reverse discrimination"--there's just discrimination based on a protected classification, such as race, color, or sex. Majority groups are protected just like minority groups.

(2) Employers do not have to go to irrational lengths to validate their selection methods. Although the tests had flaws, the court continued to demonstrate that employers simply need to follow a logical process for developing the exam to show job relatedness; the exams don't have to win any awards.

(3) Disparate treatment by a government entity in order to avoid liability for adverse impact is legal only in certain very specific instances (when there is a "strong basis in evidence"). The court has been trending for years toward "color-blind" selection decisions.

About the only thing this case really points out is employers need to be ready to use the results from whatever test they administer, barring some enormous irregularities. That, and part of a defense against an adverse impact case might be that choosing not to use the exam would have been evidence of disparate treatment (I'll grant you that one's a little confusing).

All in all--and I'm certainly not the only one who feels this way--it doesn't appear to be anything to get excited about.

Want to know more? Check out the scotuswiki page.

New newsletter

There's a new newsletter in town. It's called EEO Insight, it's published by Biddle Consulting Group, and it focuses on EEO/AA issues, including employment testing.

Check out some the topics from the first issue (December '08):


The EEOC, OFCCP, and “Systemic Discrimination”: The Rules Have Changed

Where are the Courts Today? Proving and Defending Against an “Adverse Impact” Claim: OFCCP’S New Approach to Employer Selection Systems

Five Steps to Successful AAP Goal Development Diversifying Your Organization: How to Actually Make it Happen

Claims of Employment Test Validity: Who Can You Trust?


Good stuff. You can subscribe here.

The diversity-validity dilemma (+ free articles!!)

The latest issue of Personnel Psychology has some great articles in it and right now they're free! So before you do anything else, get while the gettin's good, because normally each article will run ya $30.

So what's in there? The main attraction is a great series of articles on the "diversity-validity" dilemma, which Pyburn, Ployhart, and Kravitz in their article on the legal context, define as:

"The ability of organizations to simultaneously identify high-quality candidates and establish a diverse work force can be hindered by the fact that many of the more predictive selection procedures negatively influence the pass rates of racioethnic minority group members (non-Whites) and women."

This article is a great short read that goes over the major legal points, including adverse impact and the major court cases.

The next article, by Ployhart and Holtz, is a print-and-save type article (yes it's that good) that summarizes the various strategies employers can use to help resolve the dilemma. The article includes a couple of great tables, including one that summarizes most selection mechanisms with their corresponding criterion-related validity and d-values (pp. 155-156) and another that summarizes the various resolutions to the dilemma (pp. 158-163).

Bottom line from that article? I'll let the authors say it:

"Among the most effective strategies, the only strategy that does not also reduce validity is assessing the full range of KSAOs." (bold added)

Hallelujah. Yes, certain assessment methods tend to work better than others (e.g., structured interviews, job knowledge tests) but the best approach is plain old fashioned good practice: Start with job analysis and use the testing methods that best target the knowledge, skills, abilities, and other characteristics (KSAOs) that rise to the top. It really is pretty simple.

The third article in the series is another fabulous one, this time targeting the role that affirmative action (AA) plays in the dilemma.

In it, Kravitz provides a great overview of the basis of AA, attitudes about AA, and provides some answers to some controversial issues, including:

- Does discrimination still occur? (Answer: you bet)
- What is the economic impact of AA on target groups? (A: it's complicated)
- What is the economic impact of AA on organizations? (A: apparently very little)
- Does AA lead to stigmatization of target group members by others? (A: it can)
- Does AA lead to self-stigmatization of target group members? (A: hard to say)

The article then wraps up with some great practical recommendations, the two most important of which are strong, visible, ongoing support of management and the development of an appropriate culture.

Last but not least, don't miss the other great content in this issue, including Mount et al.'s article, Incremental validity of perceptual speed and accuracy over general mental ability and Taylor et al.'s article The transportability of job information across countries.

Now get out there and get some free content!

October '07 TIP: Alternatives and Title VII

You legal buffs out there know that under Title VII of the Civil Rights Act of 1964 (as amended in 1991) there exists a "burden shifting" framework that lays out how an employment discrimination case (hypothetically) proceeds:

1 - The plaintiff must show that the employer is using a particular employment practice (e.g., a selection test) that results in disparate (or adverse) impact against a legally protected group; if successful,

2 - The employer must show that the practice was/is job related for the position in question and consistent with business necessity; if successful,

3 - The plaintiff must show that there is an alternative employment practice (e.g., a different type of test) that would serve the employer's needs, be equally valid, and have less adverse impact and the employer refuses to adopt it. The classic case is plaintiffs suing over a written knowledge test and suggesting a personality or performance test should have been used.

You may also know that plaintiffs rarely win employment lawsuits (for many reasons, but one of which is employers are getting better at #2 above), and there seems to be a shift toward the third prong of the case--showing that there are alternative testing mechanisms out there that are equally as valid and with less adverse impact.

The October issue of the Industrial-Organizational Psychologist (TIP) contains two articles (both by individuals who have served as expert witnesses in discrimination cases) that touch on this subject and are worth a read:

Slippery slope of "alternatives" altering the topography of employment testing? by James Sharf

and

Less adverse alternatives: Making progress and avoiding red herrings by James Outtz

Also in this issue, a great analysis of the recent U.S. Supreme Court ruling in Parents v. Seattle School District by Art Gutman and Eric Dunleavy that reviews in detail the current status of affirmative action.

Links a go-go for October 1, 2007

Good reading for October 1, 2007

The new affirmative action (about schools, but lessons for employers)

2007 ILG National Conference Highlights

Don't automatically dismiss people that been fired

Court rules EEOC may proceed with discrimination case against L.A. Weight Loss

Visa and using credit scores in the hiring process

Hiring supervisors and leaders (the #1 problem of most organizations, IMHO)

Deloitte demonstrates just how creative recruiting can be

How many names does it take to get to a hire?

Who does The Gap think it is? Monster?