November '07 Issue of J.A.P.

The November 2007 issue of the Journal of Applied Psychology is full of interesting articles, including several relating to recruiting and assessment. Let's take a look:

First, a field study by Hebl et al. on pregnancy discrimination. Female confederates posed as job applicants or customers at retail stores, sometimes wearing a pregnancy prosthesis. As "pregnant" customers, they received more "benevolent" behavior (e.g., touching and over-friendliness), but as job applicants they received more hostile behavior (e.g., rudeness). The latter effect was particularly noticeable when confederates applied for stereotypically male jobs. This isn't a form of discrimination that gets as much play as others, but may be much more common than we think. My guess is a lot of people associate pregnancy with impending time off and don't focus as much on the competencies these women bring to the job.

Second, a study on faking. But wait, not faking on personality tests, faking during interviews. Levashina and Campion developed an interview faking behavior scale and then tested it with actual interviews. Guess what? Scores on the scale correlated with getting a second interview. (Looks like those classes you took on answering vaguely are going to pay off!) But wait, there's more. The authors also found that behavioral questions were more resistant to faking than situational questions (another reason to use 'em!), and follow-up questions INCREASED faking (another reason NOT to 'use em!). Other goodies in this article: over 90% of undergraduate job candidates fake during employment interviews (I assume that's just this sample), BUT, the percentage that were actually lying, or close to it, was less (28-75%).

Third, Brockner et al. provide research results that underline how important procedural fairness (justice) is. Three empirical studies demonstrated that employees judge organizations as being more responsible for negative outcomes when they experienced low procedural fairness. So when applicants or employees get bad news, they'll blame the organization even more if they feel the process used was unfair. Why do we care? Because perceptions of procedural fairness impact all kinds of things, including recruiting (e.g., how someone reacts to not getting a job) and the likelihood of filing a lawsuit (for, say, discrimination).

Fourth, Lievens, Reeve and Heggestad with a look at the impact of people re-taking cognitive ability tests. Using a sample of 941 candidates for medical school that took an admissions exam with a cognitive component, the authors found that retesting introduced both measurement and predictive bias: the retest scores appeared to be measuring memory rather than g, and predictive validity (of GPA) was eliminated. More evidence that re-testing effects are non-trivial. Pre-publication version here.

Last but definitely not least, one of my favorite topics--web-based recruitment. Allen, Mahto, & Otondo present results from 814 students searching real websites. When controlling for a student's image of the employer, job and organizational information correlated with their intention to pursue employment. When controlling for information search, a student's image of the employer was related to the intention to pursue employment, but familiarity with the employer was not. Finally, attitudes about recruitment source influenced attraction and partially mediated the effects of organizational information. What does all this mean? Don't throw your eggs into one basket--organizational image is important, but so is the specific information you have on your website about your organization and the specific job.

There's a lot of other good stuff in this volume, including articles on the financial impact of specific HRM practices, a meta-analysis of telecommuting impacts, engaging older workers, and daily mood.

September 2007 issue of IJSA

The September, 2007 issue (vol. 15, #3) of the International Journal of Selection and Assessment is out, with the usual cornucopia of good reading for us, particularly if you're into rating formats and personality assessment. Let's skim the highlights...

First, Dave Bartram presents a study of forced choice v. rating scales in performance ratings. No, not as predictors--as the criterion of interest. Using a meta-analytic database he found that prediction of supervisor ratings of competencies increased 50% when using forced choice--from a correlation of .25 to .38. That's nothing to sneeze at. Round one for forced choice scales--but see Roch et al.'s study below...

Next up, Gamliel and Cahan take a look at group differences with cognitive ability measures v. performance measures (e.g., supervisory ratings). Using recent meta-analytic findings, the authors find group differences to be much higher on cognitive ability measures than on ratings of performance. The authors suggest this may be due to the test being more objective and standardized, which I'm not sure I buy (not that they asked me). Not super surprising findings here, but it does reinforce the idea that we need to pay attention to group differences for both the test we're using and how we're measuring job performance.

Third, Konig et al. set out to learn more about whether candidates can identify what they are being tested on. Using data from 95 participants who took both an assessment center and a structured interview, the authors found results consistent with previous research--namely, someone's ability to determine what they're being tested on contributes to their performance on the test. Moreover, it's not just someone's cognitive ability (which they controlled for). So what is going on? Perhaps it's job knowledge?

Roch et al. analyzed data from 601 participants and found that absolute performance rating scales were perceived as more fair than relative formats. Not only that, but fairness perceptions varied among each of the two types. In addition, rating format influenced ratings of procedural justice. The researchers focus on implications for performance appraisals, but we know how important procedural justice is for applicants too.

Okay, now on to the section on personality testing. First up, a study by Carless et al. of criterion-related validity of PDI's employment inventory (EI), a popular measure of reliability/conscientiousness. Participants included over 300 blue-collar workers in Australia. Results? A mixed bag. EI performance scores were "reasonable" predictors of some supervisory ratings but turnover scores were "weakly related" to turnover intentions and actual turnover. (Side note: I'm not sure, but I think the EI is now purchased through "getting bigger all the time" PreVisor. I'm a little fuzzy on that point. What I do know is you can get a great, if a few years old, review of it for $15 here).

Next, Byrne et al. present a study of the Emotional Competence Inventory (ECI), an instrument designed to measure emotional intelligence. Data from over 300 students from three universities showed no relationship between ECI scores and academic performance or general mental ability. ECI scores did have small but significant correlations (generally in the low .20s) with a variety of criteria. However, relationships with all but one of the criteria (coworkers' ratings of managerial skill) disappeared after controlling for age and personality (as measured by the NEO-FFI). On the plus side, the factor structure of the ECI appeared distinct from the personality measure. More details on the study here.

Last but not least, Viswesvaran, Deller, and Ones summarize some of the major issues presented in this special section on personality and offer some ideas for future research.

Whew!

June 2007 IJSA

Yep, it's journal time again...the June 2007 issue of the International Journal of Selection and Assessment is out...

This time the articles fall into two main camps: those focused on applicant reactions to selection procedures, and those focused on personality testing.


Applicant reactions

First up, Landon and Arvey looked at ratings of test fairness by 57 individuals with graduate education in HRM. The test characteristics included validity coefficient, mean test score difference between minority and majority candidates, and test score adjustment for the minority group. Results? Fairness ratings depended almost as much on how raters rated as they did on the test characteristics! Implications? "Extensively cross validated with a global random sample of over 3 million with criterion-related validities exceeding .90!" may not be worth a hill of beans to some folk.

Next, Bertolino and Steiner with a similar study of test fairness using responses from 137 university students (ahh, where would our research be without students). Students rated 10 selection methods on 8 procedural justice dimensions. Results? Work sample tests were rated highest (as they usually are), followed by resumes, written ability tests, interviews, and personal references. All things that are expected by most job seekers. What wasn't perceived as well? Graphology--thankfully. The most important predictors of procedural justice were opportunity to perform and perceived face validity. Nothing earth shatteringly new, but some international confirmation of previous, mostly U.S., findings.

Speaking of international support for the importance of opportunity to perform and face validity among an international sample judging fairness of selection methods (say that five times fast), Nikolaou and Judge analyzed responses from 158 employees and 181 students in Greece. Methods rated highest (drumroll...): interviews, resumes, and work samples, across both groups. However, students reported more positive reactions to "psychometric tests" (i.e., ability, personality, and honesty tests) than did employees--an important distinction. Also, although there does appear to be individual differences in rating (see previous study), core self evaluations didn't appear to explain much.

In summary: Work samples and interviews--high fairness ratings and (potentially) high validity. A great combination.


Personality testing

First up in the personality section is Kuncel and Borneman's study of a new method for detecting "faking" during personality tests by looking at the particular way "fakers" respond. The authors found support for this method (sample was not described), with 20-37% of fakers identified. May not seem like a lot, but they had a false positive rate (incorrectly labeling someone a faker when they're not) of only 1% with a baserate of 56% honesty. Not too shabby. Interestingly, the "faker" pattern did not correlate with personality or cognitive ability test results.

Second, a very interesting study of the "dark side" of personality by Benson and Campbell. Using two independent samples of managers/leaders (N=1306 and 290), the authors found support for a non-linear (inverted U) relationship between "dark side" scores and assessment center ratings as well as performance ratings. So, for example, having a moderate amount of skepticism or caution is good--but too little or too much creates a problem. The instruments used were the Global Personality Inventory and the Hogan Development Survey.


Okay, there's a third category

Okay, one more study. de Mejier et al. analyzed results from over 5,000 applicants to Dutch police officer positions. The researchers were interested in rating differences when comparing ethnic minority and non-minority applicants. Results? Similar (if not more) assessment information was used to judge the two groups, but a large number of "irrelevant cues" were used to judge minority applicants. One other difference: when rating minority candidates, assessors relied more on the ratings of others. Another argument for a standardized rating process!

Can grumpy workers lead to better organizational performance?

One of the most interesting findings in industrial/organizational psychology is that the relationship between job satisfaction and job performance isn't particularly strong (although we have people looking into it).

Now, new research from Dr. Jing Zhou at Rice University may help us understand why.

Dr. Zhou gathered survey responses from 161 employees (granted, not a large sample) of a large oil-field services company and their supervisors. What she found was that people who experienced periodic bad moods tended to be more focused on detail, more analytical, and more creative--partially because they're motivated to get out of their bad mood.

Overly happy people, on the other hand, aren't likely to see potential problems until it's too late.

It depends on the job

Bob Hogan, of Hogan Assessment Systems, rightly points out that the type of person you want depends on the job. If you're hiring in advertising or product development, you might look for someone who gets agitated when confronted with a problem. On the other hand, if you're hiring for a call center, grumpy shouldn't be high on your list--instead you want the bright cheerful person who can withstand a lot of negativity.

And on the organization

Zhou and Hogan also both point out that it's not enough to have the right person--the organization must support the expression of these types of emotions and encourage change. If a frustrated person is constantly squashed or told to cheer up, those innovative ideas may never bubble to the top. There's also a tie here to perceptions of organizational justice, of which being able to voice your opinion is an important aspect.

There's grumpy, then there's GRUMPY

Of course you don't want someone who's grumpy all the time. As Zhou points out, there's not much you're able to do with someone like that. Chances are that's not a good job-person fit (although it always warrants a little investigation).

In addition, there's a distinction between grumpy and angry--after all, we want people expressing themselves appropriately, not with office furniture.

More about Dr. Zhou's research can be found in this interview.

New issue of JOOP

The December issue of the Journal of Occupational and Organizational Psychology is out so let's take a look.

First up, an article about reactions to performance feedback. Unfortunately the sample is not described (nor the measures) but the results were that leaders who received numerical/normative feedback responded better than did leaders who received text feedback--regardless of the source. Support, perhaps, for the typical practice in assessment of providing numerical scores; I'm guessing more important than type is whether feedback is provided at all, which we know is so important for feelings of procedural justice.

The next article that looks interesting is a study of teams and whether a multidisciplinary group is necessarily the best bet in all cases. The study of health care workers found that having a multidisciplinary team was related to success if defined as quality of innovations generated but not number of innovations generated.

Next up is a study of almost 52,000 school teachers in Israel. The research question? The impact of promotions on absences. The findings? The higher the person was promoted (in terms of level), the greater the decrease in absences. This impact was stronger for individuals with less tenure. Conclusion? If someone's having an abscene problem, promote them as high in the organization as you can (a little Thanksgiving Day humor for you, there).

Last but not least is a longitudinal study of 54 British symphony orchestra members (apparently a notoriously difficult group to study). Looks to me like the qualitative interview data yielded some rich information regarding the musicians' motivations and commitments. I'm always looking for good job analysis information; if you're sourcing or hiring for musicians this could give you some fresh perspectives.