Detecting liars is not a skill

One of the most popular pieces of folk wisdom is that some people are better at detecting liars than others. When it comes to selection, some people think they can tell when a candidate is lying about their history or competencies. And if the organization is conducting background checks and/or a polygraph, it becomes particularly important to detect deceptions.

Yet according to a new meta-analysis published in the most recent issue of Psychological Bulletin, we may all be pretty much the same when it comes to lie detection. In the words of the authors:

"Although researchers have suggested that people differ in the ability to detect lies, psychometric analyses of 247 samples reveal that these ability differences are minute."

Where there do appear to be differences is in being able to successfully tell a lie--some people are plain better at it than others.

The article is followed by two commentaries that are critical of this study and a reply by the author.

What about people's overall ability to detect a lie? Check out this study from 2006 by the same authors. Short answer: we're not very good.

A review of situational judgment tests

In the latest issue of Personnel Review, Dr. Filip Lievens and colleagues provide an empirical review of situational judgment tests (SJTs), focusing on studies from 1990-2007.

SJTs, sometimes referred to as low fidelity simulations, present test takers with a scenario and ask them to select the appropriate response. Candidates may be asked to select what "should" they do, what "would" they do, the best response, the worst response, or some combination of the above. Here's an example:

You have been assigned lead responsibility for two weeks in the absence of your supervisor. On your first day in this role, one of your new direct reports comes into your office and complains that they were sexually harassed by the security guard when they entered the building. They ask that the situation be kept confidential. What would be your first action in response to this situation?

1. Contact the security guard and conduct an interview to obtain all the facts.
2. Assure the direct report you will look into the situation but cannot guarantee confidentiality.
3. Contact your supervisor to obtain instruction on next steps.
4. Conduct informal interviews with your other direct reports to determine if they have been harassed.

SJTs have some great benefits, and this article points them out. First, they can be valid predictors of performance--particularly when based on job analysis. Second, they show incremental validity beyond cognitive ability and personality tests, making them a valuable addition. Third, group differences tend to be reduced compared to ability tests, particularly when the cognitive load is low. Fourth, applicant perceptions of SJTs tend to be positive. And fifth, SJTs allow you to test large candidate groups simultaneously. I would add that they allow for all kinds of scoring possibilities as well (e.g., +1 for correct response, -1 for incorrect).

SJTs aren't without drawbacks--two major ones to be exact. The first is they can be susceptible to faking, practice, and coaching effects--although how they're built plays a large role in how big these effects are. The second is that we don't always know exactly what SJTs are measuring--is it job knowledge? Personality? Cognitive ability? The authors point out that more research is needed.

Overall, a very good review of a test method that every assessment professional should have in their tool belt. You can read an in press version here.

Unproctored internet testing: Safe for some tests?

One of the biggest trends in personnel assessment is the movement toward on-line testing. Many organizations are experimenting with so called unproctored Internet testing (UIT), where candidates are allowed to take the exams whenever, and wherever, they want.

Benefits? Extremely convenient for the candidate. Less administrative resources needed by the employer.

Costs? Bye-bye exam security, hello cheating opportunities. Not only is your test out for everyone to see, but you have no real way of knowing (sans biometric verification) who is taking the test.

Some organizations have decided the benefits outweigh the risks, and a new study in the June 2008 issue of the International Journal of Selection and Assessment may provide support for their position.

In it, the authors looked at over 800 applicants from nine European countries that took a test of perceptual speed in an unproctored setting, then followed this up with a proctored parallel version. Results? Not only was there no evidence of cheating, they found the opposite effect--people did better in the proctored setting.

Now before everyone throws out their proctored exams, note that this is a type of test that might be hard to cheat on--at least in one way. Because this is a perceptual speed test, there are no "right" answers that can be looked up. It also required very quick responses. So the only way to cheat would be to have someone take the test for you. Implication: it may make more sense to use certain UITs than others.

This topic is a source of much debate in the assessment community, and there is by no means consensus on the right way to go. But studies like this help!

Take a deep breath, because there's a lot more in this issue:

- The preliminary employment interview as a predictor of assessment center outcomes (fascinating look at how the AC may only make sense for mid-range interview scorers)

- A comparison of the common-item and random-groups equating designs using empirical data (for you IRT fans out there)

- The influence of external recruitment practices on job search practices across domestic labor markets: A comparison of the United States and China

- Beneath the surface: Uncovering the relationship between extraversion and organizational citizenship behavior through a facet approach (a more nuanced look at the relationship shows extraversion can predict OCBs)

- Comparing personality test formats and warnings: Effects on criterion-related validity and test-taker reactions (another good one...personality test added predictive validity beyond ability test but no validity difference between forced-choice and Likert scales, nor between warning and no-warning conditions; forced-choice and warnings may produce negative candidate reactions)

- Applicant selection expectations: Validating a multidimensional measure in the military (describes development of a new measure of applicant perception of the selection process)

- Selecting for creativity and innovation: The relationship between the innovation potential indicator and the team selection inventory

Predicting turnover

Turnover can be caused by many things. Inadequate supervision/ leadership. Too much work. Not enough work. Insufficient career growth opportunities.

According to many surveys (e.g., salary.com's recent one), these are the types of things people report as primary motivators driving them to change employers.

But these are all factors outside of the employee. What about aspects of employees themselves that might contribute to turnover? We know that people are changing jobs more frequently these days (every 2-3 years in the U.S.), and there seems to be a persistent dissatisfaction among the Gen Xers with their careers, but what about someone's personality? Might there be individual differences between people when it comes to changing jobs?

You bet, according to a new study published in the Summer 2008 issue of Personnel Psychology. After meta-analyzing 86 studies, author Ryan Zimmerman found that personality factors, particularly emotional stability and agreeableness, play a big role in predicting turnover. Emotional stability best predicted intent to quit, while agreeableness best predicted actual turnover.

In fact, personality traits predicted turnover better than did non-self report measures such as job complexity and job characteristics.

Implications? Many initiatives designed to reduce turnover may disappoint because it's not the job, it's the person. The next time you design an exit interview or turnover study, make sure to add this reason for why the person left: It had nothing to do with the job, it was just me.

This also provides more support for using personality tests to predict important outcomes.

...

The other study in this issue we should look at provides some support for all you O*NET fans out there. You know...O*NET? The replacement for the Dictionary of Occupational Titles? Developed by the Department of Labor? A fount of job analysis knowledge? If you don't know it, you should.

Anyway, in this study, the authors used O*NET data to predict literacy requirements across a wide variety of occupations compared to scores on the national adult literacy survey (NALS). Results? O*NET did well--quite well in fact, with correlations around .80.

What does this mean? It means that occupational requirements listed in O*NET just got a big boost in terms of their validity. When it comes to job analysis, don't leave O*NET out.

Another member of the blog family

I consider this blog to be part of a fairly small family (with a few notable exceptions, e.g., Selection Matters) in that I focus on trying to put personnel psychology research into layperson terms. But I think I may have found a lost member of my blogging family.

The Association of Test Publishers (ATP) has an I-O Division, and lo and behold, they have a blog! And it's been up since January! I feel so...out of touch.

Anyway, check out some of their recent posts:

- The Validity-Diversity Dilemma (yes, we both blogged about this)

- Input Needed on "Model Guidelines" Revision

- Economic Study of Impact of Pre-Employment Assessment

So, belated welcome to the blogosphere! Like what you see? Pick up the feed.

B = f (P,E)

One of the most famous axioms in social psychology is what's sometimes called "Lewin's equation" (after the famous psychologist Kurt Lewin): behavior is a function of both the person and the environment. This equation is good to keep in mind when looking at all kinds of human behavior, including recruitment and assessment.

Research presented in the May 2008 issue of Journal of Applied Social Psychology addresses this equation. Let's take a look at it and see if helps us answer an age-old question: What's more important--the observer or what's being observed?

Tell me if this situation sounds familiar. A hiring manager insists on hiring someone based on something they saw in the person's resume (e.g., the candidate graduated from a particular college), even though the person did not do well on a structured, validated assessment. The first study shows that HR is not immune to this phenomenon. In it, HR managers were presented with two types of information about a candidate: preliminary information (like a resume) and performance on an assessment center. The managers were then asked to rate the candidate. Results? Managers were unable to exclude the preliminary information, even though they had better information (the assessment center results) in front of them.

The second article looks at the legitimacy perceptions of promotion decisions and how they relate to information on deservedness (candidate performance) and entitlement (affirmative action). Participants felt that both deservedness and entitlement were related to legitimacy, but there was a gender effect--female participants felt increased resentment when the male candidate was promoted.

The third article is a fascinating take on how people how people perceive discrimination. Specifically, the authors looked at ambiguous situations and the impact of how "prototypical" the person doing the discriminating is. What they found was that the amount of control the perceiver felt they had over discrimination in their lives moderated the influence of the prototype effect. In other words, whether a white male (the prototype) was acting in a discriminatory fashion depended a great deal on the perceiver. Like research on stress, control was found here to have a significant effect on perceptions.

So given these three articles, what's more important--the observer or what's being observed? The research above gives us a clear answer, and one that validates the wisdom of Kurt Lewin: both.

Adverse impact of assessment centers (May Applied Psych)

The May '08 issue of the Journal of Applied Psychology is out with lots of great content. Unfortunately only one is directly related to recruitment and assessment, so let's check that one out then I'll give you links to some others that look interesting.

The study is Ethnic and gender subgroup differences in assessment center ratings: A meta-analysis by Dean, Roth, and Bobko. The authors found overall d-values of .52 for Black-White differences, .28 for Hispanic-White differences, and -.19 for male-female differences. (the second group in these comparisons performs better)

The results suggest that the Black-White difference is larger than previously thought but may be a more "diversity friendly" option for Hispanics and females.

There are some other great articles in here for fans of organizational behavior, including:

Subjective cognitive effort: A model of states, traits, and time. (which, by the way, suggests another reason why conscientiousness may predict job performance)

Early predictors of job burnout and engagement.

Event justice perceptions and employees' reactions: Perceptions of social entity justice as a moderator.

Harmful help: The costs of backing-up behavior in teams.

Trust that binds: The impact of collective felt trust on organizational performance.

Stirring the hearts of followers: Charismatic leadership as the transferal of affect.

The influence of psychological flexibility on work redesign: Mediated moderation of a work reorganization intervention.

...and several more!

In search of highly skilled workers

The U.S. Merit Systems Protection Board (MSPB) has a new study out called In Search of Highly Skilled Workers. Like all MSPB studies, it's focused on the federal government. But it has a lot of good info regardless of what sector you're in.

For example:

- MSPB recommended that departments should not rely exclusively on USAJOBS since this was not the primary way people found out about jobs--word of mouth was, through friends/relatives or their supervisor/co-workers

- More and more hires are being made from the veterans program, and this results in the most white males compared to other mechanisms; this has resulted in upper level hires trending toward white males. Interestingly, the most diverse hires were made through "direct hire", which does not require rating & ranking candidates.

- Use of the competitive exam process (e.g., need to rate & rank) is less and less frequent.

- Job security was by far the top reason new hires applied for federal jobs; other highly ranked factors included mission of organization, opportunity to serve the public/do good, benefits, and opportunity to use talents/have a better job.

- OPM's television campaign is seeing some success, with increased traffic.

- Candidates are pretty lazy and/or strapped for time. About 1/3 of hires did not apply for other jobs because they would have had to re-write their narrative description of their KSAs while about 1/4 did not because they would have to have rewritten their resume (!).

- More than ½ of upper-level hires were not aware of their application status until they called for an interview/job offer (sadly, probably not that rare).

- Many new hires indicated they would have accepted their job even with a reduction in pay because of workplace flexibilities and benefits.

- Coordination of hiring managers and HR was key: 98% of managers involved with the development of assessment tools said they got the talent they needed; only 82% of managers not involved felt that way.

- Upper level hires were willing to wait about 2 months between job application and job offer--most felt anything after that was unreasonable.

- Things the feds don't do well? According to new hires, they don't deal effectively with poor performers, their leave benefits could be better, and (not surprisingly) they weren't thrilled with the pay.

- Something else federal departments can improve on: providing clear minimum qualifications that truly are the "lowest bar", and verifying that applicants have them.

Good lessons here for employers everywhere.

First issue of new SIOP journal

SIOP (the Society for Industrial and Organizational Psychology) has a new journal. It's called "Industrial and Organizational Psychology: Perspectives on Science and Practice."

The idea of the journal is to offer a couple of focal articles and then print several peer commentaries associated with each.

In the inaugural (March 2008) issue, two topics are covered.

In the first focal article, Macey and Schneider treat us to a historical and research-based view on a very hot topic, employee engagement. Here's a sample:

"The notion of employee engagement is a relatively new one, one that has been heavily marketed by human resource (HR) consulting firms that offer advice on how it can be created and leveraged. Academic researchers are now slowly joining the fray, and both parties are saddled with competing and inconsistent interpretations of the meaning of the construct."

The authors provide a great overview of the different ways of viewing engagement. The treatment is generally positive, and they sum up their view this way:

"Although engagement may best fit...as a profile model of a multidimensional construct, we see engagement as not only a set of constructs but also a tightly integrated set, interrelated in known ways, comprising clearly identifiable constructs with relationships to a common outcome."

The focal article is followed by no less than 13 commentaries from a variety of authors, both academics and consultants. The authors follow with a reply and point out that the debate over engagement is a great example of the "research-practice gap."

The second topic is assessment centers. In it, Charles Lance investigates why they don't work the way they're supposed to. Specifically, candidate ratings seem to reflect the particular exercise they're completing--not the dimensions they're supposed to be rated on. The conclusion sums up their view nicely:

"It is now time to acknowledge the last quarter century's worth of research findings and reorient assessment away from broad dimensions and toward exercise-based assessment."

The focal article is followed by ten peer commentaries by folks such as Ann Howard, Winfred Arthur, and Filip Lievens, and the authors' response.

Great, in-depth stuff for those of you out there interested in either topic.

Have teens lost the ability to write?

Have teenagers lost the ability to form a coherent sentence? It's something I've been hearing off and on the last few years from hiring managers and teachers. Generally I chalk it up to the normal generational differences, but given results from a recent survey I may have to change my tune.

The survey results come from a new study out by the Pew Internet and American Life Project--"Writing, Technology, and Teens." The researchers conducted focus groups and a national telephone survey of 700 parent-child pairs in the fall of 2007.

Here are some of the results:

• "85% of teens ages 12-17 engage at least occasionally in some form of electronic personal communication, which includes text messaging, sending email or instant messages, or posting comments on social networking sites." [They were twice as likely to send text messages as they were e-mail]

• "60% of teens do not think of these electronic texts as “writing.”"

• "50% of teens say they sometimes use informal writing styles instead of proper capitalization and punctuation in their school assignments"

• "38% say they have used text shortcuts in school work such as “LOL”"
  

• "25% have used emoticons (symbols like smiley faces ☺) in school work"
  

Wow. Color me wrong. I had no idea the practice was so rampant. But there may be a silver lining to this survey:

• "86% of teens believe good writing is important to success in life"

and they may know how to improve things:

• "82% of teens feel that additional in-class writing time would improve their writing abilities and 78% feel the same way about their teachers using computer-based writing tools"
  

So is this a big deal for recruitment and assessment? Potentially, if the work that needs to be done requires traditional, formal writing skill. But what if it doesn't? And what happens when this generation becomes managers--will they care if their subordinates use shortcuts and emoticons?

If one thing is made clear by this survey, it's not that teens don't think writing is important. It's that we may be in for a slight evolution in our written language. We may need to re-think the answer to this question: What is good writing?

Is age related to job performance?

Is there a relationship between age and job performance? It's an important question for many reasons, including the fact that claims of age discrimination appear to be on the rise. Ng and Feldman set out to better understand this issue and their meta-analysis is published in the March '08 issue of Journal of Applied Psychology.

Previous research has generally shown a weak relationship between age and job performance--at least when we look at objective measures. But the current authors set out to use a much broader array (10 to be exact) of criterion measures, including workplace aggression, safety performance, and OCBs.

So what did they find? Well, there's where things get a bit complex. Although there did not appear to be a relationship between age and several outcomes, including core task performance, creativity, and performance in training programs, it had stronger relationships with the other seven measures. In addition, age had a curvilinear relationship with core task performance and CWBs, and results varied depending on how the study was conducted.

So does age have a relationship with job performance? Like all important research questions, the answer is an emphatic it depends.


Other articles
There's quite a bit of good research in this volume, including:

- The development of a potentially useful way to predict team member performance

- A fascinating look at how frame-of-reference influences the validity of personality measures (pre-published version here)

- A discussion of the importance of the distinction between constructs (e.g., ability, personality) and methods (e.g., interviews) when comparing predictors in personnel selection

- How to test for adverse impact when your numbers are small (hint: significant testing bad, z-score good)

- Last but not least, a meta-analysis of the relationship direction between attitudes and job performance--what causes what? (hint: attitude matters...but not that much)

Too fat or too thin? You may not get hired.

Job candidates that are either too fat or too thin may have a more a difficult time getting hired than those in the middle weight ranges according to a study by Swami, et al. reported in the most recent issue of the Journal of Applied Social Psychology.

Weighting in line
The authors found that when men were asked to rate a variety of female pictures for either a management position or for providing help (N=30 and 28, respectively), they were less likely to hire or help women with body mass indices (BMI) over 30 or under 15. Those with a slender body (BMI = 19-20) were most likely to be hired or helped. This shouldn't be surprising, given that studies have consistently linked physical attributes, including weight, with employment decisions, but it's certainly a reminder to watch your biases when evaluating candidates!

Predict-ability
In another article, Truxillo et al. found a relationship between cognitive ability and the ability to accurately judge one's performance on an employment test. Using a video-based situational judgment test of customer service skills, the authors found that those with high cognitive ability were able to predict their performance while those with low cognitive ability were not. Practical implications? Providing thorough test feedback may be particularly important for candidates lower in cognitive ability as they may be more likely to be surprised (and dismayed) by the results. This means providing information prior to the test as well as afterward (e.g., how it was developed, how it is scored, how you can improve your performance).

Working IT
In a third study, Johnson, et al. found gender and ethnic group differences in how IT careers are perceived as well as in self-efficacy related to IT. Using data from 159 African- and 98 Anglo-Americans, the authors found that African American men reported higher levels of IT self-efficacy than all other groups, whereas Anglo women reported the lowest levels. In addition, Anglos had more negative stereotypes of IT professionals than did African Americans. This study had a small sample size, but the implication is that how people see their own ability related to an occupation, as well as how they perceive those in it, influences their career choices. This will in turn impact your applicant demographics as well as your recruiting success.

The rest
There are some other interesting reads in here, including:

When emotional displays of leaders may increase follower performance

How to give performance feedback

Self-perceptions of ethical behavior

Does interview coaching increase validity?

Does coaching interview-takers actually increase the predictive validity of the interview? That's certainly what the results of a recent study by Maurer et al. seem to indicate.

In the April issue of the Journal of Organizational Behavior, the researchers describe a predictive study where 146 interviewees (public safety incumbents) were provided with coaching before their situational interview. Importantly, this was coaching designed around the content of the interview and helping candidates communicate during the interview--not generic strategies like "smile a lot."

Results? Predictive validity was higher in the coached sample than an uncoached sample. Why? Well, it makes sense that if candidates are better at addressing the question--both because they have more knowledge and because they're expressing themselves better--you're getting a better view of their true knowledge (i.e., true score) and less interference (i.e., error).

Implications? If you conduct interviews as part of your hiring process (and is there someone out there that doesn't?), strongly consider providing pre-interview coaching (although I might call it something else since coaching sounds a bit suspicious). It may take a bit of your time, but it will pay off in the long run by improving your ability to predict job performance AND candidates will be happier. Big win-win.


The other study to read in this issue is by Becton et al., who looked at performance during and reactions to two selection procedures among White and African-American test-takers. The two tests were a written job knowledge test and a situational interview. The candidates were competing for promotion to Sergeant positions in a police department. Results? Both groups felt the interviews were more job related than the written test. And although African-American candidates performed worse on the written test, they felt that overall both methods were more job related (compared to Whites). Why is this important? Because some have theorized that subgroup differences are related to differences in take taking motivation. This study suggests there's something else going on.

The diversity-validity dilemma (+ free articles!!)

The latest issue of Personnel Psychology has some great articles in it and right now they're free! So before you do anything else, get while the gettin's good, because normally each article will run ya $30.

So what's in there? The main attraction is a great series of articles on the "diversity-validity" dilemma, which Pyburn, Ployhart, and Kravitz in their article on the legal context, define as:

"The ability of organizations to simultaneously identify high-quality candidates and establish a diverse work force can be hindered by the fact that many of the more predictive selection procedures negatively influence the pass rates of racioethnic minority group members (non-Whites) and women."

This article is a great short read that goes over the major legal points, including adverse impact and the major court cases.

The next article, by Ployhart and Holtz, is a print-and-save type article (yes it's that good) that summarizes the various strategies employers can use to help resolve the dilemma. The article includes a couple of great tables, including one that summarizes most selection mechanisms with their corresponding criterion-related validity and d-values (pp. 155-156) and another that summarizes the various resolutions to the dilemma (pp. 158-163).

Bottom line from that article? I'll let the authors say it:

"Among the most effective strategies, the only strategy that does not also reduce validity is assessing the full range of KSAOs." (bold added)

Hallelujah. Yes, certain assessment methods tend to work better than others (e.g., structured interviews, job knowledge tests) but the best approach is plain old fashioned good practice: Start with job analysis and use the testing methods that best target the knowledge, skills, abilities, and other characteristics (KSAOs) that rise to the top. It really is pretty simple.

The third article in the series is another fabulous one, this time targeting the role that affirmative action (AA) plays in the dilemma.

In it, Kravitz provides a great overview of the basis of AA, attitudes about AA, and provides some answers to some controversial issues, including:

- Does discrimination still occur? (Answer: you bet)
- What is the economic impact of AA on target groups? (A: it's complicated)
- What is the economic impact of AA on organizations? (A: apparently very little)
- Does AA lead to stigmatization of target group members by others? (A: it can)
- Does AA lead to self-stigmatization of target group members? (A: hard to say)

The article then wraps up with some great practical recommendations, the two most important of which are strong, visible, ongoing support of management and the development of an appropriate culture.

Last but not least, don't miss the other great content in this issue, including Mount et al.'s article, Incremental validity of perceptual speed and accuracy over general mental ability and Taylor et al.'s article The transportability of job information across countries.

Now get out there and get some free content!

Warning: Warnings may not work

Although faking is a persistent issue in personality testing, no one agrees on the best way to handle it. Some have suggested including "warning" statements in the test: letting applicants know there is a lie scale or some repercussions for false responses. But researchers are far from agreed on this strategy.

Now a new study out in the latest issue of Human Performance adds weight to the argument that warnings may not help us avoid the faking issue.

In the study, researchers had 464 participants fill out personality inventories in either a "warned" or "unwarned" condition. They then looked at the convergence of their scores with scores given to them be observers.

Results? Lower mean scores on some personality dimensions (which is often what happens) but no improvement in the convergence between self- and other-ratings. So in other words, it made a difference, but not a significant one. Implication: simply warning applicants that there are consequences for "inflating" their scores may not do much. Fortunately, it may not matter as well-constructed personality inventories (when used properly) still show useful levels of validity.


The other article in this issue related to assessment looked at the relationship between personality (specifically neuroticism), self-efficacy, gender, and performance (alternate version here). Using data from nearly 900 freshman from 10 different U.S. colleges and universities, the author found several results:

- female participants reported significantly lower levels of emotional stability and (to a lesser extent) self-efficacy--an important consideration if using these scores for selection

- there was a positive relationship between emotional stability and self-efficacy for female participants but the relationship for males was "nearly zero"

- emotional stability and gender interacted to affect self-efficacy which, in turn, affected performance (measured by GPA)

The last point is (to me) the most interesting, as it suggests that personality scores may predict performance indirectly through their relationship with other constructs (in this case, self-efficacy). This suggests another layer of analysis is needed when looking at the utility of personality tests.

Hiring a president

In my last post I wrote about how looking at pure length of experience probably isn't the best way to pick someone for a top leadership position--like President of the United States. This comes from decades of research on the relationship between work experience and job performance (interestingly Time magazine just published an article that comes to a similar conclusion using research on expertise).

So how would we go about making the most informed decision if we treated this like a hiring decision rather than an election? We've already heard from the recruitosphere on this issue. Now it's time for an assessment perspective.

We know right off the bat we need some tests to differentiate between the best candidates. And like all hiring decisions, we'd choose tests by starting out with good job analysis data. But unfortunately we don't have any.

"Waddya mean?" you say. "There's lots of experts and articles out there that have documented what makes a good president!" Ahh, yes, but that's not how we would conduct a job analysis for hiring someone. We don't just conduct a literature review, we follow the requirements of the Uniform Guidelines by, among other things, creating detailed statements describing the work to be performed and the knowledge, skills, abilities, and other characteristics (KSAOs) necessary to do it. We also have subject matter experts rate these statements on things like critical, frequency, and necessity at entry to the job.

Can you imagine getting the current incumbent in the same room with previous presidents and conducting a job analysis session?! Count me in on that meeting!

But since that's not going to happen, how can we use assessment research to inform the job of hiring a president? What do years of research tell us about hiring someone who is likely to succeed at this type of job?

Here are the tests I would consider:

- Cognitive ability test. For a complex job like president, high cognitive ability is an absolute must, and research shows ability is the #1 predictor for complex jobs. Unfortunately, we (usually) have a field of very smart applicants, so giving them an ability test might not narrow the field.

- A work sample/performance test. Each candidate is given a live scenario that's pretty close to what they'd face as president. A discussion with a world leader, acting quickly in an emergency, a press conference, or serving as mediator between two disagreeing parties. Sit back and rate the performance using pre-established rating scales.

- A structured interview. This is no softball interview with questions about favorite memories. Each candidate gets the same challenging job-related questions and we have a pre-determined rating scale with benchmarks for judging good answers.

- A job knowledge test. A comprehensive written test covering all of the topics that a president would be expected to know. If you think about it, it's rather scary to think that we hire a president without gauging their full knowledge.

- What about a personality test? This is probably the trickiest (but potentially most interesting) of all the tests. If the job analysis showed that a certain trait, measurable by a reputable instrument, related to success (and some attempts at this have been made at this) we could go forward. Research has indicated that particularly when informed by job analysis, personality tests can have useful levels of performance prediction.

- The best: All of these! Imagine an assessment center-like format where the candidates go through a day-long battery of all these tests. Police officer candidates often have to do it--why not arguably the most powerful position in the world?

What don't you see here? "Debates" that consist mostly of canned phrases, speeches to supporters, and policies that may or may not have been written by the candidate. In other words, most of what we have now. This is similar to hiring someone based purely on a resume they created.

Imagine having all the data that these tests would provide. Talk about an informed hire!

Does experience matter?

There's been a lot of talk this election season about whether experience matters when it comes to the job of U.S. president.

There's been a lot of back and forth, but I haven't heard a lot of discussion about whether there's any research related to the question. A lot of folks might be surprised to learn there's actually quite a bit of research directly related to this point. And we can use it to inform decisions like selecting a president--or any other leader for that matter.

So what does the research say? The best sources of research and analysis on this topic (e.g., this one and this one among others) have reached some general conclusions:

1. The most important conclusion is that the answer depends on how you define both experience (e.g., amount, time, type) and job performance (e.g., task, contextual) and the particular job you're looking at. There is no single answer.

2. Experience does predict job performance, but not as well as, say, cognitive ability--this is particularly true for high-complexity jobs.

3. Length of experience best predicts job performance when incumbents have relatively low amounts of experience (e.g., entry-level jobs).

4. Length of experience best predicts job performance when the job is low-complexity. At high levels of complexity it does significantly worse at predicting performance. After, say, about 5 years of experience, more doesn't seem to add anything to predicting performance.

5. Prediction is increased when we look at amount of experience performing particular tasks rather than length of experience. This makes sense--just because someone's held a job for 20 years doesn't mean they've performed the tasks you're interested in (and done them well).

So what does all this mean for, say, choosing a president? I'm afraid the answer is not simple, which is as it should be. Pure amount of experience doesn't appear to be all that important after a few years (although this is difficult to analyze since there's only one incumbent at a time!). Ultimately the question is what type of experience is important--and THAT question hasn't been answered.

For a highly complex job like president, simply looking at experience does not seem the best way to measure and predict performance. So from a personnel assessment standpoint, how would we hypothetically select a president? I'll cover that in my next post: Hiring a President.

HEXACO personality model receives more support

Although the Big 5 model of personality (generally broken into Openness to experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) has received wide empirical support, not everyone agrees it's the final word on a taxonomy for normal adult personality.

One camp believes (with reason) that honesty and humility play a consistent role in differentiating people and predicting behavior, and should be added to the Big 5 and the most recent issue of the Journal of Occupational and Organizational Psychology contains a research report that lends credence to this view.

In the article, the authors describe three results of their research:

1. Honesty-Humility was found to be distinct from the Big 5 using data from an actual selection situation.

2. Honesty-Humility scores were found to predict scores on an integrity test and a business ethical decision-making task--adding predictive value beyond the Big 5.

3. Perhaps most interestingly, this result was replicated when honesty-humility was judged by those acquainted with the target person.

Implications? This adds weight to the argument that the Big 5 may need to be expanded to include additional personality dimensions--particularly if we're using it to predict behavior such as job performance.

There are other good articles in this issue, including ones on job performance ratings, counterproductive behavior in a service environment, post-retirement work, and others.

Providing test information helps reactions

Providing applicants with pre-test information and preparation material may not help them pass, but it helps those that don't pass feel better about the process. These were the results of a study by Burns, Siers, and Christiansen reported in the March 2008 issue of the International Journal of Selection and Assessment.

Other articles include:

- An analysis of reactions to selection methods in the Netherlands. Interviews, work samples, and resumes came out on top.

- The development of a new measure of test anxiety, the Self- versus Other-referenced Anxiety Questionnaire.

- A study of validity transport statistics.

- A detailed comparison study of the FIRO-B, an assessment of interpersonal relations.

- A similar study of the PAPI-N, a personality measure.

- A study of how attitudes toward rating systems impact peer rating behavior.

- A look at how well multisource feedback on leadership competencies predicted subsequent interview performance (hint: pay attention to your supervisors and peers).

- Last but not least, a verification that "psychological hardiness" is an important facet related to success in high-stress occupations (in this case, U.S. Army Special Forces).

Links a go-go

Good reading for February 1, 2008:

U.S. Government launches photo screening tool to help employers verify employment eligibility

New Monster ads (seen the one with the sun coming up?)

Assessment trends for 2008

Increased use of video-sharing sites in U.S.

Attracting IT talent in 2008 (thank you HR World)

Do you know what you don't know you know?

HR academics vs. practitioners