Broadband adoption in U.S.: A mixed bag

A new study out by the Pew Internet and American Life Project shows that while 55% of all Americans have a broadband connection at home, up from 47% in early 2007, poorer Americans saw no increase during this time and their access rates are under 50% compared to more than 80% of upper-income Americans.

Why does this matter? In this age of bandwidth-greedy job preview videos, java-filled interactive career websites, and realistic job assessments, a high-speed connection is becoming increasingly a necessity. The good news is more than half of Americans can engage in these experiences at home. The bad news? Not only will access to some of these sites most likely have an adverse impact against certain groups (see below), this will reduce an organization's ability to draw an applicant pool that contains the most diverse backgrounds.

With that in mind, consider these findings:

* While 70% of those age 18-29 reported having broadband at home, only 50% of those age 50-64 did.

* While 57% of White respondents had broadband at home, only 43% of Black respondents did (let's see, four-fifths of 57% is...). On a more positive note, 56% of English-speaking Hispanic respondents had this access.

* 79% of those with at least a college education had home broadband access; only 40% of high school grads did.

* 60% of suburban respondents and 57% of urban respondents had this access; only 38% of rural respondents did.


"But people can always go to a library," is a response I often hear. That may be true, although not everyone lives within easy access of a library. But libraries aren't open 24/7. And many times they're busy during peak hours. And many aren't exactly a Starbucks cafe. Do you really want to create these barriers?

So what can we do about it? Here are some ideas:

* Make sure your careers site has a low-bandwidth alternative

* Consider offering a staffed on-site computer center that operates during off-peak hours (e.g., 6-8am, 5-7pm, weekends)

* Think long and hard about whether you're adopting bandwidth-hogging features because they're there or because they'll actually add value.

For more details, check out the report.

Adverse impact of assessment centers (May Applied Psych)

The May '08 issue of the Journal of Applied Psychology is out with lots of great content. Unfortunately only one is directly related to recruitment and assessment, so let's check that one out then I'll give you links to some others that look interesting.

The study is Ethnic and gender subgroup differences in assessment center ratings: A meta-analysis by Dean, Roth, and Bobko. The authors found overall d-values of .52 for Black-White differences, .28 for Hispanic-White differences, and -.19 for male-female differences. (the second group in these comparisons performs better)

The results suggest that the Black-White difference is larger than previously thought but may be a more "diversity friendly" option for Hispanics and females.

There are some other great articles in here for fans of organizational behavior, including:

Subjective cognitive effort: A model of states, traits, and time. (which, by the way, suggests another reason why conscientiousness may predict job performance)

Early predictors of job burnout and engagement.

Event justice perceptions and employees' reactions: Perceptions of social entity justice as a moderator.

Harmful help: The costs of backing-up behavior in teams.

Trust that binds: The impact of collective felt trust on organizational performance.

Stirring the hearts of followers: Charismatic leadership as the transferal of affect.

The influence of psychological flexibility on work redesign: Mediated moderation of a work reorganization intervention.

...and several more!

Are the Uniform Guidelines obsolete?

It must be Uniform Guidelines week.

In my last post I talked about how the EEOC doesn't appear to have any intentions of changing them anytime soon.

Now, in the April '08 issue of TIP Dan Biddle makes an argument in favor of their continued relevance.

Biddle provides a useful mini-history of the Guidelines and sums up nicely why we care about them:

"This document has since been used in thousands of government enforcement and judicial settings where employers have been required to demonstrate that their selection procedures causing adverse impact are sufficiently “job related” by addressing the Guidelines requirements."

He also briefly reviews the differences between the Guidelines and the other "biggies" for employers, the SIOP Principles and the APA Standards. While the two latter documents are newer and generally more thorough in their treatment of issues such as validity, when it comes to legal challenges Biddle points out that the Guidelines have been referenced in hundreds of Title VII cases while the Principles and Standards have been collectively cited less than 40 times.

Biddle ends the article with an extended treatment of validity generalization and how the topic is treated by the Guidelines and courts (not well) compared to psychologists (better). He comes down in favor of the continued relevance of the Guidelines and questions whether they need to be updated at all.

Here's my favorite quote:

"If the first burden in Title VII settings (proving adverse impact) cannot be carried using evidence solely from external locations, it seems to follow that the second burden (proving validity) should likewise not be provable using only external evidence. One can only imagine the outcry of defense attorneys if government enforcement agencies or plaintiff attorneys were permitted to transport or generalize adverse impact into a local employer based on adverse impact that occurred “at some other location.”"

Good point. And good article.

2008 PTC-NC Conference, Day 1

On March 20th and 21st I was lucky enough to attend (and present) at the 2008 PTC-NC Conference. There were several great presentations and I'm going to break the summary up into two days for ya.

The conference started off with Michael Harris, professor of International Business at the University of Missouri-St. Louis, with an update on what the courts have been saying in the area of adverse impact and employment testing. Some major points:

- Although criterion-related validity has been discussed a lot lately, Dr. Harris predicted that a content validation strategy will continue to be the most popular choice of employers.

- They're not perfect, but courts will continue to rely on the Uniform Guidelines when judging employers' efforts to validate their tests.

- Employers should be prepared to answer what alternative tests they considered before choosing the ones they did (the third prong of this type of case).

The next session was a breakout, with Chris Wright and Louis Xavier (SF State) presenting on stereotype threat while I went over the results of a demographic analysis I conducted on applicants to an on-line T&E system. Bottom line of my presentation: there were some clear demographic differences in the jobs applied for but actual instances of adverse impact (using 4/5ths rule) were rare.

Next up was another breakout, with Jim Kuthy (Biddle) presenting on AutoGOJA while Shelley Langan (CPS) presented on succession planning. I attended Shelley's, which focused on the importance of workforce planning given current demographics and provided some practical tips on how to put together a successful plan. Some key takeaways: limit succession planning to certain positions, consider inviting everyone to apply, and conduct a future-oriented job analysis as part of your planning process.

The last breakout featured an introduction to competency modeling by Nathan Ainspan (independent consultant) and a presentation on using personality testing by Shelley Langan and Howard Fortson (CPS). I attended the latter, where we had a spirited discussion of the state of personality testing and how to introduce personality testing to your selection process (hint: rhymes with job analysis). Example business measures they mentioned included the CPI, HPI, NEO, and 16-PF. They also mentioned an upcoming article by Hough & Oswald where the authors list all of the various outcomes that personality tests have been able to successfully predict.

Last but not least that day was an outstanding keynote address by James Outtz, president of Outtz and Associates and international expert on employment testing and discrimination. Dr. Outtz went over a wide range of issues related to those subjects, including the balance between validity and adverse impact (so well covered in the most recent issue of Personnel Psychology). He introduced some fascinating research that showed that while multiple choice formats showed adverse impact against African Americans and Hispanics (favoring Caucasians and Asians), the opposite was the case for multiple list (divergent) items. Perhaps most interesting was his description of a questionnaire he developed called the Job Perception Index that served as both a realistic job preview and a selection device for firefighter positions. Some great stuff from a fabulous speaker.

That's it for now--those of you that attended, feel free to comment or add! Tomorrow: Day 2.

Is age related to job performance?

Is there a relationship between age and job performance? It's an important question for many reasons, including the fact that claims of age discrimination appear to be on the rise. Ng and Feldman set out to better understand this issue and their meta-analysis is published in the March '08 issue of Journal of Applied Psychology.

Previous research has generally shown a weak relationship between age and job performance--at least when we look at objective measures. But the current authors set out to use a much broader array (10 to be exact) of criterion measures, including workplace aggression, safety performance, and OCBs.

So what did they find? Well, there's where things get a bit complex. Although there did not appear to be a relationship between age and several outcomes, including core task performance, creativity, and performance in training programs, it had stronger relationships with the other seven measures. In addition, age had a curvilinear relationship with core task performance and CWBs, and results varied depending on how the study was conducted.

So does age have a relationship with job performance? Like all important research questions, the answer is an emphatic it depends.


Other articles
There's quite a bit of good research in this volume, including:

- The development of a potentially useful way to predict team member performance

- A fascinating look at how frame-of-reference influences the validity of personality measures (pre-published version here)

- A discussion of the importance of the distinction between constructs (e.g., ability, personality) and methods (e.g., interviews) when comparing predictors in personnel selection

- How to test for adverse impact when your numbers are small (hint: significant testing bad, z-score good)

- Last but not least, a meta-analysis of the relationship direction between attitudes and job performance--what causes what? (hint: attitude matters...but not that much)

Does interview coaching increase validity?

Does coaching interview-takers actually increase the predictive validity of the interview? That's certainly what the results of a recent study by Maurer et al. seem to indicate.

In the April issue of the Journal of Organizational Behavior, the researchers describe a predictive study where 146 interviewees (public safety incumbents) were provided with coaching before their situational interview. Importantly, this was coaching designed around the content of the interview and helping candidates communicate during the interview--not generic strategies like "smile a lot."

Results? Predictive validity was higher in the coached sample than an uncoached sample. Why? Well, it makes sense that if candidates are better at addressing the question--both because they have more knowledge and because they're expressing themselves better--you're getting a better view of their true knowledge (i.e., true score) and less interference (i.e., error).

Implications? If you conduct interviews as part of your hiring process (and is there someone out there that doesn't?), strongly consider providing pre-interview coaching (although I might call it something else since coaching sounds a bit suspicious). It may take a bit of your time, but it will pay off in the long run by improving your ability to predict job performance AND candidates will be happier. Big win-win.


The other study to read in this issue is by Becton et al., who looked at performance during and reactions to two selection procedures among White and African-American test-takers. The two tests were a written job knowledge test and a situational interview. The candidates were competing for promotion to Sergeant positions in a police department. Results? Both groups felt the interviews were more job related than the written test. And although African-American candidates performed worse on the written test, they felt that overall both methods were more job related (compared to Whites). Why is this important? Because some have theorized that subgroup differences are related to differences in take taking motivation. This study suggests there's something else going on.

2008 PTC-NC Conference

The Personnel Testing Council of Northern California (PTC-NC) is hosting its annual conference on March 20-21 in Concord.

They've lined up quite an agenda with some great presentations. Here's a sample:

- Disparate Impact and Employment Testing by Michael Harris

- Situational Judgment Testing by Jim Outtz

- Personality Assessments by Bob Hogan

Oh yeah, and I'll be doing a session on demographic application patterns and adverse impact of an on-line T&E system.

For more information contact Jerimiah Honer at jhoner@spb.ca.gov

October '07 TIP: Alternatives and Title VII

You legal buffs out there know that under Title VII of the Civil Rights Act of 1964 (as amended in 1991) there exists a "burden shifting" framework that lays out how an employment discrimination case (hypothetically) proceeds:

1 - The plaintiff must show that the employer is using a particular employment practice (e.g., a selection test) that results in disparate (or adverse) impact against a legally protected group; if successful,

2 - The employer must show that the practice was/is job related for the position in question and consistent with business necessity; if successful,

3 - The plaintiff must show that there is an alternative employment practice (e.g., a different type of test) that would serve the employer's needs, be equally valid, and have less adverse impact and the employer refuses to adopt it. The classic case is plaintiffs suing over a written knowledge test and suggesting a personality or performance test should have been used.

You may also know that plaintiffs rarely win employment lawsuits (for many reasons, but one of which is employers are getting better at #2 above), and there seems to be a shift toward the third prong of the case--showing that there are alternative testing mechanisms out there that are equally as valid and with less adverse impact.

The October issue of the Industrial-Organizational Psychologist (TIP) contains two articles (both by individuals who have served as expert witnesses in discrimination cases) that touch on this subject and are worth a read:

Slippery slope of "alternatives" altering the topography of employment testing? by James Sharf

and

Less adverse alternatives: Making progress and avoiding red herrings by James Outtz

Also in this issue, a great analysis of the recent U.S. Supreme Court ruling in Parents v. Seattle School District by Art Gutman and Eric Dunleavy that reviews in detail the current status of affirmative action.

September '07 issue of J.A.P.

The September 2007 Journal of Applied Psychology is out and it's got some research we need to look at...

First, Hogan, Barrett, and Hogan present the results of a study of over 5,000 job applicants who took a 5-factor personality test on two separate occasions (6 months apart). Only 5% of applicants improved their score on the second administration, and scores were equally as likely to change in a negative direction as a positive one. The authors suggest that given these results, faking on personality measures is not a significant issue in real-world settings. Comment: This is faking defined as improving your score to match the job, not faking as misrepresenting yourself consistently over time. Also, I can't help but think of the recent article by Morgeson et al. in Personnel Psych where they argued that faking isn't the issue; it's the low validities we should be concerned about.

Next, De Corte, Lievens, and Sackett present a procedure designed to determine predictor composites (i.e., how much each testing method should be "worth") that optimize the trade-off between validity and adverse impact. The procedure is tested with various scenarios with positive results. You can actually download the executable code here and an in-press version of the article is here (thank you, Drs. De Corte and Lievens!).

Speaking of validity, the next article of interest is by Newman, Jacobs, and Bartram, and looks at the relative accuracy of three techniques for estimating validity and adverse impact (local validity studies, meta-analysis, and Bayesian analysis). The authors describe which method is optimal in different conditions, using measures of cognitive ability and conscientiousness as predictors. They even toss in recommendations for how to estimate local parameters.

Next, a fascinating and useful study of counterproductive work behaviors (CWBs; things like theft, sabotage, or assault) by Roberts, Harms, Caspi, and Moffitt that tackles the issue from a developmental perspective. Using data from a 23-year longitudinal study of 930 individuals, the authors found that individuals diagnosed with childhood conduct disorder were more likely to commit CWBs as young adults. On the other hand, criminal convictions occurring at a young age were unrelated to CWBs demonstrated later on. Job conditions and personality traits had their own impact on CWBs, above and beyond background factors. Great stuff, especially for those of you with a particular interest in biodata and/or background checks.

Last but not least, a study of person-organization (P-O) fit by Resick, Baltes, and Shantz. Using data from 299 participants in a 12-week internship program, the authors found that the relationship between P-O fit on the one hand and job satisfaction, job choice, and job offer acceptance on the other depends on the type of fit (needs-supplies vs. demands-abilities) as well as the conscientiousness of the individual. Good food for thought when thinking about P-O fit, a consistently popular concept.

Honorable mention: This meta-analysis by Humphrey, Nahrgang, and Morgeson of 259 studies that investigated work design impacts on worker attitudes and behaviors. Think behavior is determined solely by individual ability and disposition? Ya might want to take a gander at this study; it'll change your tune. A great reminder that satisfaction and performance are the result of both the individual and his/her work environment. Also available here.

Georgia-Pacific fined by OFCCP for using literacy test

In a display of "See? It's not just the EEOC you need to worry about", the U.S. Department of Labor's Office of Federal Contract Compliance Programs (OFCCP) has fined the Georgia-Pacific Corp. nearly $750,000.

Why? During a "routine audit of the company's hiring practices", the OFCCP discovered that one of Georgia-Pacific's paper mills was giving job applicants a literacy test that resulted in adverse impact against African-American applicants (saw that one coming a mile away). The $749,076 will be distributed to the 399 applicants who applied for a job while the mill was using the test.

The test required applicants to read "bus schedules, product labels, and other "real-life" stimuli." The OFCCP determined that the test was not backed by sufficient evidence of validation for the particular jobs it was being used for.

The company defended itself by saying it promotes heavily from within and wanted workers to be able to move around easily.

A sensible policy, but completely irrelevant in terms of defending the legality of a test. In fact it works against an employer, since (as one of the attorneys points out) you're in effect testing people for higher-level positions, which is a no-no.

Several attorneys are quoted in the article, and they mention the importance of the Uniform Guidelines, which really only apply when a test has adverse impact, as in this case. It does make me wonder what sort of validation evidence G-P collected (if any)...

Note: the article states incorrectly that "all federal contractors" are subject to OFCCP rules. Actually only certain ones are, and details can be found here.

Hat tip.

Links a go-go for 8-24-07

Good reading for August 24, 2007:

OFCCP issues final regulations implementing Jobs for Veterans Act of 2002 (job banks for postings listed here), and...

OFCCP also posts interim guidance on use of race and ethnic categories (direct link here)

Are you sure you know where your hires are coming from? (hint: beware drop-down boxes)

Jobmatchbox does the 50 top recruiting blogs

Interview questions you can ask--and those you can't (includes simplistic video!)

Tracking adverse impact

Favorite defense motions in limine for employment cases

Does harassment training lead to more lawsuits? (hat tip)

The housing market and its relationship to recruiting

New regulations on no-match letters: Ho-hum?

September 2007 issue of IJSA

The September, 2007 issue (vol. 15, #3) of the International Journal of Selection and Assessment is out, with the usual cornucopia of good reading for us, particularly if you're into rating formats and personality assessment. Let's skim the highlights...

First, Dave Bartram presents a study of forced choice v. rating scales in performance ratings. No, not as predictors--as the criterion of interest. Using a meta-analytic database he found that prediction of supervisor ratings of competencies increased 50% when using forced choice--from a correlation of .25 to .38. That's nothing to sneeze at. Round one for forced choice scales--but see Roch et al.'s study below...

Next up, Gamliel and Cahan take a look at group differences with cognitive ability measures v. performance measures (e.g., supervisory ratings). Using recent meta-analytic findings, the authors find group differences to be much higher on cognitive ability measures than on ratings of performance. The authors suggest this may be due to the test being more objective and standardized, which I'm not sure I buy (not that they asked me). Not super surprising findings here, but it does reinforce the idea that we need to pay attention to group differences for both the test we're using and how we're measuring job performance.

Third, Konig et al. set out to learn more about whether candidates can identify what they are being tested on. Using data from 95 participants who took both an assessment center and a structured interview, the authors found results consistent with previous research--namely, someone's ability to determine what they're being tested on contributes to their performance on the test. Moreover, it's not just someone's cognitive ability (which they controlled for). So what is going on? Perhaps it's job knowledge?

Roch et al. analyzed data from 601 participants and found that absolute performance rating scales were perceived as more fair than relative formats. Not only that, but fairness perceptions varied among each of the two types. In addition, rating format influenced ratings of procedural justice. The researchers focus on implications for performance appraisals, but we know how important procedural justice is for applicants too.

Okay, now on to the section on personality testing. First up, a study by Carless et al. of criterion-related validity of PDI's employment inventory (EI), a popular measure of reliability/conscientiousness. Participants included over 300 blue-collar workers in Australia. Results? A mixed bag. EI performance scores were "reasonable" predictors of some supervisory ratings but turnover scores were "weakly related" to turnover intentions and actual turnover. (Side note: I'm not sure, but I think the EI is now purchased through "getting bigger all the time" PreVisor. I'm a little fuzzy on that point. What I do know is you can get a great, if a few years old, review of it for $15 here).

Next, Byrne et al. present a study of the Emotional Competence Inventory (ECI), an instrument designed to measure emotional intelligence. Data from over 300 students from three universities showed no relationship between ECI scores and academic performance or general mental ability. ECI scores did have small but significant correlations (generally in the low .20s) with a variety of criteria. However, relationships with all but one of the criteria (coworkers' ratings of managerial skill) disappeared after controlling for age and personality (as measured by the NEO-FFI). On the plus side, the factor structure of the ECI appeared distinct from the personality measure. More details on the study here.

Last but not least, Viswesvaran, Deller, and Ones summarize some of the major issues presented in this special section on personality and offer some ideas for future research.

Whew!

Class certified in Novartis gender discrimination suit

Bad news for Novartis Pharmaceuticals.

On July 31, 2007 Judge Gerald Lynch of the Southern District of New York granted class certification status to "[a]ll women who are currently holding, or have held, a sales-related job position with [Novartis] during the time period July 15, 2002 through the present."

The plaintiffs are seeking $200 million in compensatory, nominal, and punitive damages, claiming that Novartis discriminates against women in a variety of ways, including compensation, promotions, performance appraisals, and adverse treatment of women who take pregnancy leave.

The case in instructive for us because of how the judge viewed expert opinion in this case. One of the plaintiffs' experts noted that Novartis' performance evaluation system was flawed because ratings were subject to modification by higher-level supervisors and because ratings had to fit into a forced distribution. In addition, appeals by employees went to either the manager who made the original rating or an HR person with no real authority to change ratings.

Another plaintiffs' expert noted that male sales employees are 4.9 times more likely to get promoted to first-line manager than female sales employees. In addition, 15.2% of male employees were selected to be in the management development program compared to only 9.1% of eligible female employees--a difference of 6.0 standard deviations.

What these statistics really signify and whether the plaintiffs end up ultimately winning the suit is anyone's guess. The important thing here is to keep in mind that what you may think is a logical way to make promotion decisions may look "subjective" to others and riddled with potential for bias to enter the equation.

Bias (and risk) can be reduced by implementing practices such as:

1 - Having raters undergo intensive training, including a discussion of potential biases and several "dry runs" of the process.

2 - Having a standardized rating form with clear benchmarks based on an analysis of job requirements.

3 - Considering carefully the use of a "forced distribution" system. If you do use one, make sure raters and ratees alike understand why--and how--this is being done.

4 - Making performance in the current job only part of the promotional criteria--give applicants a chance to show their stuff through ability tests, work sample tests, personality tests, and the like.

5 - Taking complaints seriously. If someone believes there is an opportunity for abuse of the system, investigate.

6 - Track, track, track those applicant flow statistics, including selection into training programs. Uncover discrepancies before they uncover you.

7 - Get HR involved--not just as gatekeepers but as partners. Hold HR accountable for providing best practices.

8 - If you have something like a management academy, make the criteria for entry transparent and have a point person for questions.

You can read the order here, and read more analysis of the case here.

Police depts relax hiring standards

In response to serious recruiting challenges, many U.S. police departments are "lowering" their standards for hiring.

The reasons behind the shortage are many, including a strong job market, the Iraq war, and a high number of retirements.

Departments are using whatever means they have at their disposal, including upping their advertising. Case in point: while driving down 880 the other day in Oakland, CA, I noticed a sign promoting the $69,000 starting salary for Oakland Police Officers (and people wonder why it's hard to hire in the Bay Area).

The article cited above describes many steps departments are taking, some of which may initially seem like cause for concern. Let's take a look at them:

1. Forgiving minor criminal convictions, particularly old ones. If someone got busted 10 years ago for doing Ecstasy in college, and hasn't been in trouble since, is that still relevant?

2. Relaxing the 2-year college degree requirement, or allowing experience substitutions. I'm familiar with some research indicating a relationship between college education and officer performance, but if an officer has relevant experience (and performed well), this seems like a wash.

3. Raising the age limit. Age and job performance has been a hot topic in I/O psychology for a long time. While there are some declines over age (e.g., working memory), my reading is that they aren't practically significant in most situations. And we're talking about raising the limit to 40 or 44, not 85.

4. Relaxing fitness requirements. To me this comes back to plain 'ol validation. Granted, it's not always easy to determine where a pass point should be set (do they have to run 300 meters in 55 seconds or 56 seconds?), but do the study. Find out where a reasonable point would be. Run the numbers. See if it makes sense.

A lot of the concerns that go along with these changes--hiring people with low integrity, hiring people physically or mentally unable to perform the job--can be mitigated with good assessment, such as memory tests, physical ability testing, integrity testing, and reference and background checks.

Overall, I think this is a good thing--minimum qualifications (MQ) are often barriers to employment for certain ethnicities, women, and individuals with disabilities. And the situation is even worse when they aren't based on any rigorous study of the necessity for the MQs to being with.

On the other hand, I have heard anecdotally that similar changes in standards for U.S. Army recruits has resulted in more challenges for training.

What do you think--big deal or not?

2007 SIOP Conference: Highlights, Part 3

This is the fourth in a series of posts about the 2007 SIOP Conference. In Part 1 I talked about some of the new products out there and in Part 2 and Part 3 I reviewed some of the research that was presented. This post continues that review...


Employment interview structure and discrimination litigation verdicts: A quantitative review

Pool, McEntee, and Gomez analyzed 31 federal court cases from 1990 to 2005 (27 claims of disparate treatment, 7 of adverse impact) to see if there was a relationship between the amount of interview structure and verdicts in employment discrimination cases. Most cases (73%) were brought under Title VII and involved promotional decisions (65%). Race discrimination was the most common allegation (47%) and the vast majority of cases (84%) involved a single plaintiff. For both types of claims, the strongest factors associated with a victory for the defendant (employer side) was having interviewers that were familiar with job requirements and having a guide for conducting the interview. In disparate treatment claims, defendants were more likely to prevail if they also had standardized questions and identical interviews for each applicant. In disparate impact cases, defendants fared better when they had evidence of validity (which makes sense given the burden shifting in these cases). Similar results to Williamson et al.’s 1997 study, but good data to have—see, we’re not just saying standardize those interviews because we’re sadistic HR folks.

Recruiting through the stages: Which recruiting practices predict when?

This meta-analysis by Uggerslev and Fassina of 101 studies looked at the impact that various “recruitment predictors” (e.g., job-person fit, job/organizational attraction) had on various outcome criteria (e.g., job pursuit intention, acceptance intentions). Results depended somewhat on the criterion, but perceived fit between the individual and the job/organization was across-the-board the strongest predictor. The only criterion that matched perceived fit was job characteristics, which tied for predicting acceptance intentions. The strength of the correlations varied, from a low of .15 between perceived fit and job choice to .47 between perceived fit and recommendation intentions. So how do we use this? The authors suggest efforts to increase the appearance of a good fit between the values of goals of applicants and those of the organization may pay off (I'm thinking, say, by focusing on aesthetics and message customization or clearly indicating what you’re looking for).

Meta-analysis on the relationship between Big Five and academic success

Okay, so it's not directly about recruitment or assessment, but it's still interesting. The title pretty much says it all--the presenters (Trapmann, Hell, Hirn, and Schuler) were looking here at the relationship between Big Five personality traits and academic success. Results? As you might expect, it depends what you mean by "success." Neuroticism was related to academic satisfaction (hey, that's why they're neurotic, right?) while Conscientiousness correlated with grades and retention. The other three factors (Extraversion, Openness, and Agreeableness) were not related to success.

That's probably the end of my review of 2007 SIOP presentations, unless I manage to obtain more presentations. Stay tuned for reviews from the upcoming IPMAAC conference!

EEOC Meeting Focuses on Employment Testing and Screening

On Wednesday, May 16th, the U.S. Equal Employment Opportunity Commission (EEOC) held a meeting to discuss issues relating to employment testing and screening, including the relevant laws enforced by the EEOC (e.g., Title VII, ADA, ADEA).

Several issues were discussed, including potential problems with specific screening methods (e.g., cognitive ability tests, credit checks), how the EEOC can better serve employers, and steps employers need to take in order to meet professional and legal guidelines (e.g., gathering validity evidence, investigating alternative methods with less adverse impact). Not for the first time, speakers emphasized that the Uniform Guidelines on Employee Selection Procedures need to be updated.

Speakers included EEOC staff members, plaintiffs in two of the more discussed recent cases (EEOC v. Dial Corp. and EEOC v. Ford Motor Co.), attorneys, and professionals in the field of assessment, including James Outtz and Kathleen Lundquist, who have frequently been retained as expert witnesses in employment discrimination cases.

Said Richard Tonowski from the EEOC:

"A mature technology of testing promises readily-available methods that serve as a check against both traditional forms of discrimination as well as the workings of unconscious bias. If that is the promise, then the threat comes from institutionalizing technical problems not yet fully addressed, the undermining of equal employment opportunity under the guise of sound selection practice, and the unintended introduction of new problems that will require resolution to safeguard test-takers and test-users."

Personality testing was mentioned prominently as an increasingly common practice among employers, but it appears (contrary to my earlier fears) that the focus was on those tests that could be considered "medical tests" under the ADA (such as the original MMPI), which leaves out many products, including the HPI, 16PF, and PCI.

Hopefully I'll have the slides from the presentation to post soon. In the meantime, check out this excellent summary from an attendee, and you can view the EEOC press release here. Statements of the speakers, along with their bios, can be found here, and it looks like the meeting transcript will be available there as well.

EEOC targets...personality tests?

So I'm looking at some information about the EEOC's new E-RACE Initiative, which they describe as a systemic effort designed to ensure workplaces are free from race and color discrimination, and I come across this quote:

"Studies reveal that some employers make selection decisions based on names, arrest and conviction records, employment and personality tests, and credit scores, all of which may disparately impact people of color."

The citations for this sentence include a study on criminal records and one on names.

This triggers a couple questions for me...

1) What is the difference between an "employment" test and a personality test? Is this just redundancy or was it intentional?

2) More importantly, where is the evidence that personality tests have disparate impact? The research I'm familiar with indicates that differences between subgroups are relatively small.

As far as I know, cognitive ability tests are still the biggest 'offender' when it comes to racial differences in test scores (although this can be reduced by focusing on aspects of cognitive ability, such as short-term memory). Seems like this is where the EEOC would want to focus, along with background checks?

Updated: FedEx settlement

Please note: This is an updated post that corrects an inaccurate fact I had posted, namely that the test in question was PSI's Basic Skills Test. This was not true. I relied on information from another blog, which turns out to have been incorrect. I sincerely apologize about the error on my part. No harm was intended.

FedEx Corp. has agreed to settle a class action employment discrimination claim (Satchell v. FedEx Express) for $54.8 million filed on behalf of black and Hispanic workers who claimed systematic discrimination in performance evaluations, promotion, compensation, and discipline throughout the Western Region. This is one of the larger settlements for this type of claim (the grand prize of $508 million goes to Hartman v. Powell)

The part of the suit involving a test claimed that it resulted in disparate impact against class members and FedEx failed to show they had the validation evidence to back it up. The settlement requires that FedEx discontinue the use of the test for courier, ramp transport driver, and customer service positions. The suit claimed that 86% of white employees had passed the test, compared to 47% of black employees and 62% of Latino employees, a clear violation of the "4/5ths rule."

Unfortunately at this point we don't have any details about the validation efforts (or lack thereof) that FedEx Express took to ensure that the test was being used appropriately. What this case does emphasize is the need for organizations to make sure they can stand behind their selection practices--particularly ones that result in adverse impact.

Some questions to ponder:

Do you track your applicant flow/adverse impact statistics (basic example here)?
If so, what do you do with the results?
If not, well...take this as a warning.

Materials

- Motion for class certification

- Plaintiff's reply memorandum in support of class certification

- Order approving class certification

- Notice to potential class members

Media articles about this case can be found (among other places) here and here.

George Lenard has discussed this case here, here, and here. Michael Harris discusses it here.

Human Performance, Vol. 20, #1: The EQ-i:S

March journal madness continues with the latest issue of the journal Human Performance.

There's really only one article in here related to selection, but it's an interesting one, so let's take a look.

In this study of 229 students from an southeastern U.S. university, Grubb and McDaniel looked at the constructs measured and fakability of the Emotional Quotient Inventory Short Form (EQ-i:S), a popular proposed measure of emotional intelligence.

The bottom line results:

- Participants were able to "fake" their scores, raising them substantially (by .83 standard deviations). Not particularly surprising as it's fairly well established that non-cognitive measures can be "faked." (What's not clear is whether it matters...)

- The two "screens" built into the EQ-i:S to try to identify fakers correctly identified only 31% of the fakers.

- EQ-i:S scores were predicted by the Big Five measure with a multiple correlation of .79. This result, say the authors, "casts doubt on the construct of emotional intelligence as operationalized in the EQ-i:S."

But there's some additional goodness in this article, largely because the authors also had the participants take the Wonderlic Personnel Test (WPT) and a measure of the Big Five personality factors (IPIP):

- WPT scores were correlated strongly with only one Big Five factor--Openness.

- The correlation between WPT scores and gender was small.

- Black-White score differences on the Big Five factors were small.

- Gender was correlated with Big Five scales, but the nature varied depending on the condition (honest or faking).

- The ability to fake on the EQ-i:S was a function of cognitive ability and personality (mostly agreeableness).

There were other articles, one of which I'll discuss over at HR Coal.

New issue of IJSA--vol. 15, #1

The March journal madness continues with the latest issue of the International Journal of Selection and Assessment.

There's quite a lot in this issue (and plenty for you personality testing junkies), so let's take a look at some of the articles...

The main section

1. Hulsheger and colleagues report a meta-analysis of the operational validity of cognitive ability (general mental ability, g, whatever ya wanna call it) in Germany. After analyzing data from 54 articles and reports, the authors report a population correlation of .467 with training success and .534 with job performance. Interestingly (and counter to other research I've seen), they found the relationship was higher for low-complexity jobs.

2. Next, a study of situational judgment tests (SJTs) by O'Connell and colleagues. Using data from seven U.S. manufacturing companies (total N of around 1,000), the authors had findings in three areas. First,they found a mean Black-White difference on the SJT of .38 and a gender difference of -.27 (favoring females). Second, for task performance they found that the SJT added incremental validity to a cognitive ability test or a personality test, but not if both were already being used. Last, for contextual performance, they found the SJT added validity only if a cognitive ability test was being used as the sole instrument. Good stuff, and kudos as always to Dr. Michael McDaniel for making this (and other articles) available through his website.

3. Next up, a very interesting concurrent study of 154 customer service employees from DeGroot and Kluemper. Why very interesting? Because they looked at the impact of "vocal attractiveness." Results? Vocal attractiveness correlated with both situational interview scores and job performance. AND, two Big 5 factors (agreeableness and conscientiousness) predicted performance more strongly for people with more attractive voices. Interesting! I knew all those vocal lessons would pay off.

Let's see...what else looks good...

4. How about another concurrent study of person-organization (P-O) fit? McCulloch and Turban looked at the incremental validity of a measure of P-O fit, defined as the match between managers' description of the work culture with incumbent work preferences. Results? The measure added significant incremental variance over a cognitive ability measure in terms of predicting turnover (important given the historically high levels for these jobs) but was not related to job performance. A great illustration (along with #2 above) of the importance of the criteria being studied.

5. A study of Hartman's Color Code Personality Profile by Ault and Barney. If you don't know much about this test (I didn't), it classifies people into four colors--Red (motive: power), Blue (intimacy), White (peace), and Yellow (fun). Anyhoo, the authors state this is the first psychometric research on the highly popular instrument (don't even get me started), and found "high" test-retest reliability and support for the instrument measuring "some" personality traits. They note, however, that the instrument has high error variance and suggest caution when using it at the individual level. This is one of those studies you'll want to have in your back pocket when someone comes up to you and says, "Hey, I read about this cool personality test on my flight. Let's use that from now on."

Speaking of personality testing...

This issue has a special section on personality. Let's see what we've got:

6. A study that looks into response distortion on personality tests. Berry, Page, and Sackett found support among a sample of 261 managers for enhanced prediction of job performance when self-deceptive enhancements (SDE) were accounted for, but not impression management (IM) scores. Why is this important? Because most research on "faking" personality tests has looked at IM--intentionally inflating your scores to score better on the test. SDE, on the other hand, happens when you honestly believe you are greater than you are. I think I need this article.

7. The brain stimulation continues with a meta-analysis by Connolly, Kavanagh, and Viswesvaran. The authors were looking at the relationship between self-ratings and observer-ratings of Big 5 personality dimensions. Results? Correlations in the .5-.6 range, suggesting each contributes substantial unique variance. Oh yes, and duration of acquaintance had a large moderating effect, but source of observer ratings did not. I may need this one too.

8. Last but not least (yeah, I skipped a couple, but I know your e-mail Inbox is filling up as we speak), Smithikrai found, in a sample of 2,518 Thais employed in seven different occupations, that the Big 5 factor of neuroticism was significantly negatively correlated with job success across the board, while extraversion and conscientiousness showed significant positive correlations. Conscientiousness was the only trait to predict success across occupations (sounds familiar!).

That's all folks! Enjoy your weekends!