New York City settles discrimination case for $21M

Eliot Spitzer isn't the only one in New York that's paying for mistakes.

New York City has agreed to settle an employment discrimination case that dates back to 1999 for $21 million. This case is particularly interesting given its focus on recruiting practices.

The class action lawsuit was filed by black and Hispanic employees of the Department of Parks and Recreation who complained that the department was illegally discriminating in its promotion and assignment practices.

Specifically:
"The plaintiffs complained that they were bypassed by promotions because of a recruiting program Mr. Stern [the former Parks commissioner] had started to recruit young graduates of elite colleges — nearly all of them white — to fill positions in the agency."

Of the recruiting program, Mr. Stern said:
"The program was to get young college graduates to work long hours at low salaries. The problem was you couldn’t [get] black graduates to work for $22,000 or $25,000, either because they had loans or were offered better jobs by companies that wanted them."

What could the City have done to prevent this situation? Given the actual statistics (the article states 40 of the 179 hired were black or Hispanic), this was likely more about the fairness and perception of the process rather than hiring numbers. A different communication strategy and engagement with current employees likely would have gone a long way toward preventing the complaints.

Note that this lawsuit is separate but related from a one filed in 2002 (that was settled in 2005) which claimed that the department was illegally discriminating by favoring whites for promotion. That suit contended that:

"Time and again...the Parks Department failed to follow any objective guidelines for determining promotions and filling management positions, failed to post notices of job openings, and ‘’rarely, if ever'’ conducted the required interviews for vacancies."

As part of the current settlement agreement, the City agreed to:

"train interviewers to ensure that employees who apply for promotions are treated fairly and objectively; and to examine the process by which managers are selected in the future."

Good lessons for employers everywhere.

The diversity-validity dilemma (+ free articles!!)

The latest issue of Personnel Psychology has some great articles in it and right now they're free! So before you do anything else, get while the gettin's good, because normally each article will run ya $30.

So what's in there? The main attraction is a great series of articles on the "diversity-validity" dilemma, which Pyburn, Ployhart, and Kravitz in their article on the legal context, define as:

"The ability of organizations to simultaneously identify high-quality candidates and establish a diverse work force can be hindered by the fact that many of the more predictive selection procedures negatively influence the pass rates of racioethnic minority group members (non-Whites) and women."

This article is a great short read that goes over the major legal points, including adverse impact and the major court cases.

The next article, by Ployhart and Holtz, is a print-and-save type article (yes it's that good) that summarizes the various strategies employers can use to help resolve the dilemma. The article includes a couple of great tables, including one that summarizes most selection mechanisms with their corresponding criterion-related validity and d-values (pp. 155-156) and another that summarizes the various resolutions to the dilemma (pp. 158-163).

Bottom line from that article? I'll let the authors say it:

"Among the most effective strategies, the only strategy that does not also reduce validity is assessing the full range of KSAOs." (bold added)

Hallelujah. Yes, certain assessment methods tend to work better than others (e.g., structured interviews, job knowledge tests) but the best approach is plain old fashioned good practice: Start with job analysis and use the testing methods that best target the knowledge, skills, abilities, and other characteristics (KSAOs) that rise to the top. It really is pretty simple.

The third article in the series is another fabulous one, this time targeting the role that affirmative action (AA) plays in the dilemma.

In it, Kravitz provides a great overview of the basis of AA, attitudes about AA, and provides some answers to some controversial issues, including:

- Does discrimination still occur? (Answer: you bet)
- What is the economic impact of AA on target groups? (A: it's complicated)
- What is the economic impact of AA on organizations? (A: apparently very little)
- Does AA lead to stigmatization of target group members by others? (A: it can)
- Does AA lead to self-stigmatization of target group members? (A: hard to say)

The article then wraps up with some great practical recommendations, the two most important of which are strong, visible, ongoing support of management and the development of an appropriate culture.

Last but not least, don't miss the other great content in this issue, including Mount et al.'s article, Incremental validity of perceptual speed and accuracy over general mental ability and Taylor et al.'s article The transportability of job information across countries.

Now get out there and get some free content!

Warning: Warnings may not work

Although faking is a persistent issue in personality testing, no one agrees on the best way to handle it. Some have suggested including "warning" statements in the test: letting applicants know there is a lie scale or some repercussions for false responses. But researchers are far from agreed on this strategy.

Now a new study out in the latest issue of Human Performance adds weight to the argument that warnings may not help us avoid the faking issue.

In the study, researchers had 464 participants fill out personality inventories in either a "warned" or "unwarned" condition. They then looked at the convergence of their scores with scores given to them be observers.

Results? Lower mean scores on some personality dimensions (which is often what happens) but no improvement in the convergence between self- and other-ratings. So in other words, it made a difference, but not a significant one. Implication: simply warning applicants that there are consequences for "inflating" their scores may not do much. Fortunately, it may not matter as well-constructed personality inventories (when used properly) still show useful levels of validity.


The other article in this issue related to assessment looked at the relationship between personality (specifically neuroticism), self-efficacy, gender, and performance (alternate version here). Using data from nearly 900 freshman from 10 different U.S. colleges and universities, the author found several results:

- female participants reported significantly lower levels of emotional stability and (to a lesser extent) self-efficacy--an important consideration if using these scores for selection

- there was a positive relationship between emotional stability and self-efficacy for female participants but the relationship for males was "nearly zero"

- emotional stability and gender interacted to affect self-efficacy which, in turn, affected performance (measured by GPA)

The last point is (to me) the most interesting, as it suggests that personality scores may predict performance indirectly through their relationship with other constructs (in this case, self-efficacy). This suggests another layer of analysis is needed when looking at the utility of personality tests.

Hiring humor

A little Dilbert humor for your Friday...

Increasing self-selection is fine, but you can get carried away

And what will you do when the perfect candidate shows up?

(Side note: I swear that second cartoon rings a bell)

Hiring a president

In my last post I wrote about how looking at pure length of experience probably isn't the best way to pick someone for a top leadership position--like President of the United States. This comes from decades of research on the relationship between work experience and job performance (interestingly Time magazine just published an article that comes to a similar conclusion using research on expertise).

So how would we go about making the most informed decision if we treated this like a hiring decision rather than an election? We've already heard from the recruitosphere on this issue. Now it's time for an assessment perspective.

We know right off the bat we need some tests to differentiate between the best candidates. And like all hiring decisions, we'd choose tests by starting out with good job analysis data. But unfortunately we don't have any.

"Waddya mean?" you say. "There's lots of experts and articles out there that have documented what makes a good president!" Ahh, yes, but that's not how we would conduct a job analysis for hiring someone. We don't just conduct a literature review, we follow the requirements of the Uniform Guidelines by, among other things, creating detailed statements describing the work to be performed and the knowledge, skills, abilities, and other characteristics (KSAOs) necessary to do it. We also have subject matter experts rate these statements on things like critical, frequency, and necessity at entry to the job.

Can you imagine getting the current incumbent in the same room with previous presidents and conducting a job analysis session?! Count me in on that meeting!

But since that's not going to happen, how can we use assessment research to inform the job of hiring a president? What do years of research tell us about hiring someone who is likely to succeed at this type of job?

Here are the tests I would consider:

- Cognitive ability test. For a complex job like president, high cognitive ability is an absolute must, and research shows ability is the #1 predictor for complex jobs. Unfortunately, we (usually) have a field of very smart applicants, so giving them an ability test might not narrow the field.

- A work sample/performance test. Each candidate is given a live scenario that's pretty close to what they'd face as president. A discussion with a world leader, acting quickly in an emergency, a press conference, or serving as mediator between two disagreeing parties. Sit back and rate the performance using pre-established rating scales.

- A structured interview. This is no softball interview with questions about favorite memories. Each candidate gets the same challenging job-related questions and we have a pre-determined rating scale with benchmarks for judging good answers.

- A job knowledge test. A comprehensive written test covering all of the topics that a president would be expected to know. If you think about it, it's rather scary to think that we hire a president without gauging their full knowledge.

- What about a personality test? This is probably the trickiest (but potentially most interesting) of all the tests. If the job analysis showed that a certain trait, measurable by a reputable instrument, related to success (and some attempts at this have been made at this) we could go forward. Research has indicated that particularly when informed by job analysis, personality tests can have useful levels of performance prediction.

- The best: All of these! Imagine an assessment center-like format where the candidates go through a day-long battery of all these tests. Police officer candidates often have to do it--why not arguably the most powerful position in the world?

What don't you see here? "Debates" that consist mostly of canned phrases, speeches to supporters, and policies that may or may not have been written by the candidate. In other words, most of what we have now. This is similar to hiring someone based purely on a resume they created.

Imagine having all the data that these tests would provide. Talk about an informed hire!

Does experience matter?

There's been a lot of talk this election season about whether experience matters when it comes to the job of U.S. president.

There's been a lot of back and forth, but I haven't heard a lot of discussion about whether there's any research related to the question. A lot of folks might be surprised to learn there's actually quite a bit of research directly related to this point. And we can use it to inform decisions like selecting a president--or any other leader for that matter.

So what does the research say? The best sources of research and analysis on this topic (e.g., this one and this one among others) have reached some general conclusions:

1. The most important conclusion is that the answer depends on how you define both experience (e.g., amount, time, type) and job performance (e.g., task, contextual) and the particular job you're looking at. There is no single answer.

2. Experience does predict job performance, but not as well as, say, cognitive ability--this is particularly true for high-complexity jobs.

3. Length of experience best predicts job performance when incumbents have relatively low amounts of experience (e.g., entry-level jobs).

4. Length of experience best predicts job performance when the job is low-complexity. At high levels of complexity it does significantly worse at predicting performance. After, say, about 5 years of experience, more doesn't seem to add anything to predicting performance.

5. Prediction is increased when we look at amount of experience performing particular tasks rather than length of experience. This makes sense--just because someone's held a job for 20 years doesn't mean they've performed the tasks you're interested in (and done them well).

So what does all this mean for, say, choosing a president? I'm afraid the answer is not simple, which is as it should be. Pure amount of experience doesn't appear to be all that important after a few years (although this is difficult to analyze since there's only one incumbent at a time!). Ultimately the question is what type of experience is important--and THAT question hasn't been answered.

For a highly complex job like president, simply looking at experience does not seem the best way to measure and predict performance. So from a personnel assessment standpoint, how would we hypothetically select a president? I'll cover that in my next post: Hiring a President.

OFCCP Webinar on March 6

On Thursday, March 6, the U.S. Department of Labor's Office of Federal Contract Compliance Programs (OFCCP) will be having a webinar. The webinar will start with an overview of the agency's policymaking agenda but the main event will be an extended Q&A session.

The presenters will be OFCCP Deputy Director David Frank and Lynn Clements, Acting Director of OFCCP's Division of Policy, Planning, and Program Development.

Reasons to care:

- They oversee several laws relating to recruiting and hiring

- They go after employers (PDF file)

- A little something called the Internet Applicant Rule

You can sign up here.

HEXACO personality model receives more support

Although the Big 5 model of personality (generally broken into Openness to experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) has received wide empirical support, not everyone agrees it's the final word on a taxonomy for normal adult personality.

One camp believes (with reason) that honesty and humility play a consistent role in differentiating people and predicting behavior, and should be added to the Big 5 and the most recent issue of the Journal of Occupational and Organizational Psychology contains a research report that lends credence to this view.

In the article, the authors describe three results of their research:

1. Honesty-Humility was found to be distinct from the Big 5 using data from an actual selection situation.

2. Honesty-Humility scores were found to predict scores on an integrity test and a business ethical decision-making task--adding predictive value beyond the Big 5.

3. Perhaps most interestingly, this result was replicated when honesty-humility was judged by those acquainted with the target person.

Implications? This adds weight to the argument that the Big 5 may need to be expanded to include additional personality dimensions--particularly if we're using it to predict behavior such as job performance.

There are other good articles in this issue, including ones on job performance ratings, counterproductive behavior in a service environment, post-retirement work, and others.

Feds to increase fines for hiring undocumented workers

Just in case you needed another reason to fill out the I-9 when you hire someone, the federal government announced on Friday it will soon increase fines 25% against employers that hire undocumented workers.

Need another reason? Okay, Homeland Security and DOJ are working to increase criminal penalties for "the most egregious" offenders. By the way, this is one of the very few instances where there are CRIMINAL penalties associated with failing to cover all your bases when hiring someone. Yup, this means jail time.

New HR Tests feature: Shared items

I read a lot of blogs and other websites. I try to include links to the most interesting things here, but I don't always get around to it.

So I've created a Shared Items site using Google Reader that's available here. If you're interested in receiving links to articles I found interesting, it is now available to you either through the webpage, by signing up to receive e-mails, or by using the feed.

If you use Google Reader, it's a simple click to add the feed to your list. If you don't, copy this shortcut and add it to your feed reader. I've also added the ability to sign up via e-mail and feed on my main page.

I'll still be referencing content here but I'm going to try to stick as much as possible to original content and analysis. This is just a way for those of you out there, like me, who have an unhealthy addiction to information.

Thanks for reading!

The Acting Quotient

Think about your favorite actor or actress. Think about their performances and the different characters they've played.

Now think about what you know about that person given their performances:

- How quickly would they learn?
- How good of a team player would they make?
- What are their customer service skills like?
- Would they show up to work punctually?

Do you have any idea what the answers to these questions are? Probably not. That's because they're good at what they do--modifying their behavior to fit different roles.

Why do I bring this up? Because lately I've been thinking about how much stock people place in the most common screening mechanisms--applications, resumes, and interviews. These selection mechanisms all allow enormous opportunities for candidates to alter their behavior to fit what they think you want to hear, or to downright fib. Another up-and-coming selection method, on-line training and experience (T&E) measures, offer similar opportunities.

This is one reason why testing doesn't perfectly predict performance--because what you see isn't always close to what you get. What's worse, we tend to think we can spot the fibbers--which just plain isn't true. We're not nearly as good at most things as we think we are.

A person's real performance can be seen as true score or typical performance, and researchers have looked into the extent to which people self-inflate. The academic term for this is impression management. But for our purposes let's just think about an individual's ability to pull the wool over the employer's eyes--their Acting Quotient (AQ).

What causes someone's AQ to be higher?

- The person is a good actor or actress. It's just a natural skill. They know what their strengths are, and the play to them. They're good at reading you, and they modify their behavior to fit the subtle cues you're giving them. In fact people can vary their acting method depending on what kind of question you ask!

- They're very motivated to get your job. This could be good (my skills match your needs), this could be bad (I'm about to get fired).

- They understand what you're looking for. Again, this could be good (they did their homework) or less impressive (they know someone who works for you already).

Okay, now the important part. What can we do to mitigate the impact of someone's AQ?

- Ask difficult technical questions whenever possible. Not all jobs lend themselves to this, but whenever you can ask questions that require job knowledge and aren't easily predictable.

- Conduct extensive reference checks. Rely heavily on off-list checks. Talk to co-workers and customers, not just supervisors.

- Triangulate and be patient. Make application/resume review and interviews a small portion of your selection process. Make no decisions until all the information is in.

- Have a bias for work sample/performance tests. Make someone show you they can do things--not just tell you they can.

Now on the bright side, many if not most people (including myself) aren't very good at acting. Sure, I can put a positive spin on my accomplishments, but I'm not very good at making things up out of whole cloth. Which is good, because it means when I interview, like most people, what you see is generally what you get. But that doesn't mean there aren't Academy Award nominees out there, and it doesn't mean we can't do better at hiring the right person.

New MSPB study has gems

A new study out by the U.S. Merit Systems Protection Board (MSPB), "Attracting the next generation: A look at federal entry-level new hires," while focused on the federal government, has good information for all of us. The study of recent federal hires uses data from over 1,000 survey participants as well as personnel records. Some of the insights include:

- Both those under 30 and those 30+ were most likely to find out about the job through a friend or relative. On the other hand, those 30+ were more likely to use USAJOBs and agency web sites, whereas those under 30 (not surprisingly) were more likely to discover jobs through school job fairs or placement offices. In addition, those under 30 were more likely to use "non-government job sites."

- The most common forms of assessment used were (mostly unstructured) interviews and reviews of transcripts. Narrative statements, written exams, and job simulations were (unfortunately) used much less often--despite evidence of their superior ability to predict job performance.

- Agencies also relied heavily on the "point method" for reviewing training and experience--i.e., the more training/experience you have, the higher your score. This despite evidence that the correlation between experience and performance is complex and peaks fairly quickly, while education does not generally predict performance very well. Not to mention that they make little sense when hiring for entry-level jobs.

- There were very few differences between age groups when it came to what applicants wanted in a job. For both the under 30 and 30+ groups, salary and benefits came out tops. The only significant differences came when looking at pensions (more popular among those 30+) and health insurance and tuition reimbursement (more popular among those under 30). Job security was also mentioned by both groups as a big draw to the federal government.

- Those under 30 were much less likely to predict staying with the feds for their entire career (although still an impressive 45%), and more likely to predict eventually moving to the private sector.

- "Length of the process" was by far the biggest obstacle faced by applicants in their pursuit of a federal job.


MSPB recommends federal agencies make several improvements to their recruiting and assessment efforts, including:

- Use more predictive assessment tools instead of relying on training and experience measures.

- Use a balanced set of recruitment practices to reach all the various segments of the job market. MSPB points out that conducting a proper job analysis and identifying true minimum qualifications can help cut down the number of unqualified applicants.

- Market what is important to applicants. For federal agencies this means job security, benefits, and the ability to make a difference.

- Evaluate the hiring process and remove any obstacles that are unnecessarily lengthening the process.

- Avoid stereotyping based on generational assumptions, since the data indicate substantial similarities between age groups in what they want out of a job.

Again, good information for all of us. There's a lot more information in the report.

2008 PTC-NC Conference

The Personnel Testing Council of Northern California (PTC-NC) is hosting its annual conference on March 20-21 in Concord.

They've lined up quite an agenda with some great presentations. Here's a sample:

- Disparate Impact and Employment Testing by Michael Harris

- Situational Judgment Testing by Jim Outtz

- Personality Assessments by Bob Hogan

Oh yeah, and I'll be doing a session on demographic application patterns and adverse impact of an on-line T&E system.

For more information contact Jerimiah Honer at jhoner@spb.ca.gov

Providing test information helps reactions

Providing applicants with pre-test information and preparation material may not help them pass, but it helps those that don't pass feel better about the process. These were the results of a study by Burns, Siers, and Christiansen reported in the March 2008 issue of the International Journal of Selection and Assessment.

Other articles include:

- An analysis of reactions to selection methods in the Netherlands. Interviews, work samples, and resumes came out on top.

- The development of a new measure of test anxiety, the Self- versus Other-referenced Anxiety Questionnaire.

- A study of validity transport statistics.

- A detailed comparison study of the FIRO-B, an assessment of interpersonal relations.

- A similar study of the PAPI-N, a personality measure.

- A study of how attitudes toward rating systems impact peer rating behavior.

- A look at how well multisource feedback on leadership competencies predicted subsequent interview performance (hint: pay attention to your supervisors and peers).

- Last but not least, a verification that "psychological hardiness" is an important facet related to success in high-stress occupations (in this case, U.S. Army Special Forces).

Poking fun at personality tests

Scott Adams pokes fun at personality tests in today's Dilbert (link here after Sunday), but there is some truth behind the jabs.

For instance:

- It's too easy to make and distribute employment tests, and no governing or approval body to assist consumers in separating the rare wheat from the mostly chaff (e.g., "SIOP Certified").

- Tests can make it easy to label people--heck, I bet half of you reading this know what your Myers-Briggs type is. And that's not good--both because no test is perfect and because no one should be known simply by a test score.

- Often the questions on personality tests frustrate the taker--the scales aren't continuous and sometimes force what seem like bizarre comparisons (e.g., "Do you prefer pancakes or walks in the park?").

These are problems that have plagued tests, particularly personality tests, for a long time. What are we going to do about them?

IPMAAC Sponsor/Exhibitor Opportunities

Information has been posted on how you can become a sponsor or exhibitor at the upcoming IPMAAC conference in Oakland, California.

IPMAAC is the leading professional organization focused on employee assessment and selection. The annual conference attracts a wide variety of professionals, including individuals at all levels of government, professors and students, and consultants and other professionals from the private sector.

Click here to see a sample of presentations from previous years.

Using personality tests to prevent turnover

Turnover is costly and a major headache for most employers that becomes an even bigger problem when the labor market tightens. Can different hiring practices help solve the problem?

Perhaps, says Texas A&M professor Ryan Zimmerman, whose work in the area has recently been profiled in various places, including the national press, recruiting websites, and SIOP (article no longer available except cached version).

So what has Dr. Zimmerman found? We'll know more when his meta-analysis research is published in an upcoming Personnel Psychology issue, but in a nutshell his research found that three aspects of the Big 5 have been linked to turnover:

- Agreeableness
- Conscientiousness
- Emotional stability

Zimmerman argues that selecting for people based on these qualities is just as powerful as, say, how the job is designed.

Some things to consider while we wait for the research to be published:

- Survey after survey show that quality of supervision is a powerful retention factor (check out the Gallup 12 to see an example). This suggests selecting good supervisors may be an equally powerful factor.

- Other research, including some by Zimmerman himself, suggests person-organization (P-O) fit may also be a significant contributing factor to turnover. P-O fit is a combination of both the person and the environment, so personality would be just one side of the equation.

- Many people forget there is positive and negative turnover--not all turnover is bad because there are some people who simply aren't a good fit for the job. Simply finding a correlation between one factor and turnover is the beginning of the story, not the end.

- Average job tenure, particularly among younger generations, is around 2-3 years. So is turnover really what we should be focusing on?

Stay tuned here for more details when the research gets published.

Links a go-go

Good reading for February 1, 2008:

U.S. Government launches photo screening tool to help employers verify employment eligibility

New Monster ads (seen the one with the sun coming up?)

Assessment trends for 2008

Increased use of video-sharing sites in U.S.

Attracting IT talent in 2008 (thank you HR World)

Do you know what you don't know you know?

HR academics vs. practitioners

Et tu, HR?

It's easy to give out advice. HR does it all the time.

But how well does HR practice what it preaches?

In the most recent issue of the U.S. Merit System Protection Board's (MSPB) newsletter, Issues of Merit, in an article titled "Taking our own advice", the author describes attempts by MSPB to ensure they are walking the walk when it comes to hiring procedures.

So what did they find?

1) Their job announcements weren't all that attractive. They contained jargon and too much information. Perhaps more importantly, they didn't "sell" the job.

Solutions: Job description was refined and rewritten so qualifications were easier to understand, got rid of excess information, and made easier to read. In addition, they added a job preview component that helps applicants decide if the job is a good fit.

2) They were using questionnaires as an initial screen that had low validity.

Solution: Questionnaires were replaced with an accomplishment record, which they hope (and research suggests) will better predict who will succeed in the job.

3) They were using the "rule of three" for external hires which limited their ability to consider a broad candidate group.

Solution: "Rule of three" replaced with category rating, which allows them to consider more candidates.

4) Recruitment methods weren't as broad as they could be.

Solution: MSPB worked with OPM to feature their jobs prominently through USAJOBS, the federal government's online job posting site. In addition they made greater efforts to actively seek out qualified candidates whose resumes were in USAJOBS. Finally, they used professional organizations to help advertise their opportunities.

An honest review of recruitment and assessment procedures in any organization will undoubtedly result in areas for improvement. Kudos to MSPB for following their own advice.

January '08 issue of J.A.P.

The January 2008 issue of the Journal of Applied Psychology is out. Unfortunately there are only three articles directly related to recruitment and assessment, but they're pretty good ones, so let's dive in.

First up, a Monte Carlo investigation of the impact of faking on personality tests by Komar, et al. "What is a Monte Carlo investigation?" you may ask. Essentially it's when researchers use computers to simulate data scenarios rather than collecting actual data from participants/subjects/victims. Anyway, the researchers looked at the impact on the criterion-related validity (as measured by supervisory ratings) of conscientiousness scores adjusting for various "faking" scenarios. They found that the validity is impacted by a variety of factors, most notably proportion of fakers, magnitude of faking, and the relationship between faking and performance. Another shot across the bow of self-report personality inventories, methinks, although the debate will no doubt continue!

Next a fascinating study of motherhood bias in both expectations and screening decisions by Heilman and Okimoto. The researchers found a bias against both male and female parents when it came to anticipated job commitment, achievement striving, and dependability, although anticipated competence was uniquely low for mothers and seemed to be the major contributing factor to lowered expectations and screening recommendations. An unfortunate reminder that these factors do matter and something to watch out for. The results are reminiscent of negative behavior toward "pregnant" women found in a previous study.

Finally, Zyphur, Chaturvedi, and Arvey present a discussion of job performance. They address two subjects: the impact of past performance on future performance and individual differences in performance trajectories. Analyzing past literature, the authors note that performance feedback influences future performance directly and different individuals do have different latent performance trajectories, which has big implications for selection. Why? Because many assessment techniques (e.g., T&Es, behavioral interviews) rely on an general assumption that more experience equals better performance. This study adds ammunition to those that argue that assumption has serious flaws (or at least is overly simplistic).

In addition to these three, you may find the following interesting as well:

Challenging conventional wisdom about who quits: Revelations from corporate America. (great stuff for those of you interested in retention)

Effectiveness of error management training: A meta-analysis. (for all you trainers out there)

Effects of task performance, helping, voice, and organizational loyalty on performance appraisal ratings. (for those interested in performance ratings)

U.S. Army recruiting challenges continue

Despite aggressive (and creative) recruiting tactics, and a $4B budget, the U.S. Army failed to meet its recruiting goal in 2007, according to a report by the National Priorities Project.

Even worse than missing the numbers, the Army continues to see a change in who is being recruited. For the third consecutive year they failed to get sufficient numbers of recruits with high school diplomas, which the Army reports is the single best predictor of successful completion of a first term of enlistment.

In addition, historically, the Army had a goal of at least 67% of recruits landing in at least the 50th percentile of the Armed Forces Qualification Test (AFQT, which tests word knowledge, paragraph comprehension, arithmetic reasoning, and mathematics knowledge; more details here). Since 2005 this number has fallen and in 2007 it was 60.8%.

The Army puts educational attainment and AFQT scores together for a measure of "quality." According to the report:

A ‘high quality’ recruit is one who scores at or above the 50th percentile on the AFQT, and who is tier I (has a regular high school diploma or better). The DoD strives to have all recruits be ‘high quality’ as these recruits will be more likely to complete contracted enlistment terms and perform better in training and on the job.

Unfortunately the percentage of "high quality" recruits continues to drop. In 2005 it was 56.2%. Last year it was 44.6%.

The report includes state-by-state tables as well as a discussion of what the implications of this recruiting challenge are for the Army. Worth a look.

Tips for hiring more individuals with disabilities

The U.S. Equal Employment Opportunity Commission (EEOC) recently issued a report titled Improving the participation rate of people with targeted disabilities in the federal workforce.

Long title aside, the report offers all employers some concrete suggestions for increasing the hiring rate of individuals with disabilities, a chronically underemployed segment of the population. These recommendations include some that you probably have already thought of (establish a task force, train your managers, recruit broadly) but also include some you may not have:

- Ensure hired individuals can perform the essential functions of the position (so you don't set people up for failure)

- Use a panel interview to minimize bias of a single individual (which we know is a best practice for interviews generally)

- Use clear language and avoid jargon on job announcements (something we should be doing anyway)

- Encourage people with disabilities to self-identify and ensure confidentiality (your numbers may be artificially low)

There's a lot of good information and resources in the report that go beyond the federal workforce. Definitely worth checking out.

Wanted: Chief Magic Official

You've heard of Chief Knowledge Officer, Chief Fun Officer, and even Chief Evangelist.

But Chief Magic Official?

One guess as to who would be filling that job title.

If you guessed Disney, go to the head of the class.

Yes, Disney is out to hire it's first "CMO" who will appear periodically to grant "dreams" to guests at Disney Parks in Anaheim and Orlando this year. Disney's put together a great recruiting website that includes:

- A Magic Aptitude Test (M.A.T.) that you can take to see if you qualify--you have to take a look at it just to see the pencil you'll be using for the test (by the way, I passed and am apparently similar to Mickey Mouse)

- A great job preview video

- A creative job description and statement of qualifications

In a new twist, applications must include a video resume that will be voted on online. The top three vote getters will be invited (along with three guests) to Walt Disney World resort for further vetting.

So will this work? Probably. It is Disney, after all, who doesn't usually have too much trouble attracting candidates. And some research indicates applicants are more attracted to creative job titles.

But whatever happens, you gotta admire their creativity!

Marsalis jazzes up new EEOC PSAs

The U.S. Equal Employment Opportunity Commission, as part of its E-RACE Initiative, has produced two new public service announcements with renowned jazz musician Wynton Marsalis focusing on discrimination and equal opportunity. They're short and to the point, and you can see them here.

SIOP Conference Registration Open

Registration is now open for SIOP's Annual Conference, to be held April 10-12 in San Francisco.

Cost for early registration (before 2/29): $140 for members, $330 for nonmembers, $85 for students.

Workshops will be held on the 9th and are $400 for members, $650 for nonmembers, and this includes two workshops.

Good things about the SIOP conference: tons of presentations and lots of technical information.

Bad things about the SIOP conference: tons of presentations and lots of technical information.

December '07 IJSA, Part 2

In a previous post I described one of the most interesting articles in the December '07 issue of IJSA (the International Journal of Selection and Assessment (IJSA)). In this post I'll go over the rest of the issue.

First up, Carless & Imber with a study of interviewer characteristics. Using a sample of 450 graduate students, they found that personal aspects of the interviewer, such as warmth, friendliness, and humor, as well as job-related aspects such as job knowledge and general competence, had effects on how attracted the person was to the job as well as their likely job choice intentions. Not only that, aspects of the interviewer impacted how anxious the interviewee was. Lesson: Much like recruiters, make sure people doing the interviewing are trained and the type of people you'd like representing your organization!

Next, LaHuis, MacLane, and Schlessman with a study of reapplication among 542 applicants to a U.S. government position. Focusing on the 9% that didn't get the job but reapplied the following year, the authors found that "opportunity to perform" played a significant role in reapplication behavior. Lesson: If you want people to reapply, give them the chance to show their strengths.

Third, Carless and Wintle with a study of what attracts applicants to particular job opportunities--specifically looking at flexible career paths and who's doing the recruiting. Participants were 201 "young, inexperienced job seekers" who completed a questionnaire. Results? Flexible career paths were a big attraction (compared to traditional career paths) but recruiter background (HR or external agency) made no difference, in line with previous research that's found that recruiter personality is the key rather than things like background or demographics.

Next, Hermelin, Lievens, and Robertson conducted a meta-analysis of assessment center scores. Based on 26 studies (N=5,850) the authors found a corrected correlation of .28 between AC scores and subsequent supervisory ratings, which they hypothesize is lower than the true value due to range restriction of assessment center scores (and lower than the corrected value of .37 that Schmidt & Hunter reported). Alternate version (if the link gets fixed) here.

Heinsman, et al. provide the next study, which looks at how psychologists view the makeup and measure of competencies, still a hot topic in HR circles. Using data from over 900 applicants who participated in a 1-day selection process, they examined the relationship between competencies including Thinking, Feeling, and Power with traditional measures of cognitive ability, personality, and assessment center performance. Results? To assess Thinking, psychologists in this study relied upon measures of cognitive ability (makes sense!). To assess Feeling, they used interview simulation scores as well as personality tests scores. Finally, when analyzing Power, they relied mostly on personality scores.

Last but not least, Lievens and Anseel with a fascinating study of creating alternate versions of computerized in-baskets. The alternate version used a very similar situation that people had to respond to, but altered the context for 10 of the 20 items. Results? No significant difference in overall in-basket score between the two forms. So if you're looking to duplicate your in-baskets, check this out! Oh, and there's an alternate version of the article here.

Okay, I might as well tell you about the other two articles, because you might be interested. One is on self and peer ratings and one is on survey non-respondents (hint: it's not the star performers that aren't responding). Both have interesting results, so check 'em out!

December '07 IJSA, Part 1

The December '07 issue of the International Journal of Selection and Assessment (IJSA) has a lot of great content, so I decided to split it up between two posts. This post will be devoted to one of the studies because it's such a hot topic--web-based recruitment.

In this study, Van Hoye and Lievens gathered data from 108 applicants for head nurse positions in Belgium and looked at how they responded to two types of web-based information. The first was employee testimonials ("click here to see our happy employees talk about their jobs!"), a very common approach used by many organizations including Enterprise, Google, and the U.S. government.

The second was employer recommendations via word of mouth; in this case because it was over the web, the authors dubbed it word-of-mouse ("hey, did you hear about the opportunity at XWZ? They're a great place to work.").

Results? Well, there are several, and they all link back to source credibility.

First, word-of-mouse was a more powerful attractant than testimonials. Why? It appears the testimonials are seen as obviously controlled by the organization (and therefore possibly misleading), whereas word-of-mouse is seen as more credible. However, if you're going to use testimonials, those that focus on the individual were much more powerful than those focusing on the organization. The reverse was true for word-of-mouse: a focus on the organization was more powerful than on the employees (which underlines the importance of employer brand).

The big result is that considering all four combinations, word-of-mouse information about the organization was the clear winner in terms of attracting applicants. So efforts designed to increase the likelihood of people spreading the good word about your organization are likely to pay off. How do we do that? The authors offer some focus areas:

- image management
- campus recruitment
- building relationships with key opinion leaders (e.g., career counselors, class presidents)
- employee referral programs
- internships

I would add another: make sure your employees like what they do! Happy employees are hands down one of your most effective recruiting techniques (assuming you can't offer millions in stock options).

This is a keeper, folks, and there's a version here thanks to Dr. Lievens.

HR World's Top HR Blogs

It's always nice to ring in the new year with some good news, which is why I was pleased to see this blog on HR World's Top 25 HR Blogs.

Hope you and yours enjoyed a very happy holiday season!

Content of the year

At the end of this, the first full year of this blog's existence, I decided to take a look back at 2007 and give you my Top 5 most popular posts of the year:

1. Jobfox plays matchmaker (there continues to be significant interest in Jobfox and their non-traditional approach to matching applicants with employers)

2. Reliability and validity--it's okay to Despair(.com). Whether it's the statistics words or Despair, I'll never know. But people sure like those little posters (and remember, you can make your own).

3. Personality testing basics (Part 2). As you can see from the sidebar survey, folks continue to be very interested in personality testing.

4. Wonderlic revises their cognitive ability test. Wonderlic, one of the oldest and most famous testing companies, continues to generate interest.

5. Checkster and SkillSurvey automate reference checking. There's further development to be had, but I do believe these tools could be a boon to HR and supervisors alike.

Okay, so enough about me. What about what everyone else is writing about? Here are my nominations for content of the year:

1. Morgeson et al. fired a shot across the bow of personality testing with their piece in Personnel Psychology that resulted in multiple, shall we say, not so thrilled responses. I don't know where this debate is going (although I suspect alternate measurement methods will play a part) but it sure is fun to watch!

2. There were some great books I came across this year. Particular props for Understanding statistics, Evidence-based management, and Personality and the fate of organizations. Yes, they were all published in 2006...are you saying I'm behind?

3. Dineen et al.'s great little piece of research on P-O fit and website design in the March issue of J.A.P. that I wrote about here. Take a look at your career website with these results in mind.

4. The Talent Unconference was a big success, and I'm very thankful that many of the presentations were videotaped; I put up links to some of them here

5. McDaniel et al.'s meta-analysis of situational judgment test instructions. Not only is this a great piece of research, it's (still) free!

So what about my New Years wish from last year? I'm still waiting. Although if people search databases like Spock eventually get up enough steam...perhaps I'll get my wish?

Here's to hoping 2008 is filled with interesting and useful things!

Help Wanted

Sometimes the ad writes itself

Links a go-go

A smörgåsbord of good reading for this holiday week, 2007:

More evidence that IQ has a environmental component

Teens creating Internet content in greater numbers

New law increases mandatory retirement age for commercial pilots

Wednesday best day of week to send out recruitment e-mail (hat tip)

How to improve your corporate career website (Part I, Part II, Part III, Part IV)

Ford, et al. wrap up EEOC lawsuit over written exam

Free ATS! No, really! (Okay, there are several installation steps)

...and last but not least, for those of you doing major shopping this week:

Recruiting at the mall (I've wondered why more orgs don't do this!)

Call for Proposals for 2008 IPMAAC Conference

You're invited to submit a proposal for the 2008 IPMAAC Conference, to be held on June 8-11. The theme of next year's conference is Building Bridges: Recruitment, Selection, and Assessment and the conference will be in Oakland, California in the beautiful San Francisco Bay Area.

IPMAAC (IPMA-HR Assessment Council) is the leading organization for HR assessment and selection professionals interested in such diverse topics as job analysis, personality testing, recruiting, and web-based assessment.

IPMAAC members are psychologists, attorneys, consultants, management consultants, academic faculty and students, and other professionals who share a passion for finding and hiring the right person for the job. Whether you're from the public sector, private sector, or non-profit, everyone is welcome.

Please consider submitting a proposal for the 2008 conference--it could be a presentation, a tutorial, a panel discussion, a symposium, or a pre-conference workshop. If you have something you'd like to share, don't be shy.

Even if you don't submit a proposal, plan on attending the conference. It's a great opportunity to meet knowledgeable and passionate people and learn what everyone else is doing out there. Some presentations at last year's conference, held in New Orleans, included:

Succession planning and talent management

Using personality assessments in community service organizations

Legal update

Potholes on the road to Internet Applicant compliance

See you there!

Links a go-go

Good reading for December 14, 2007:

Why your company needs to be on Facebook

Target Corp. to pay $510K for race discrimination in application process

Check out the U.S. Office of Personnel Management's virtual conference

Monster launches website targeting Hispanic applicants

SIOP's December, 2007 newsletter

Is your organization green? Candidates want to know.

Using social networking sites to reach out to entry-level hires

The importance of background checks (white paper)

HR.com's Virtual Conference

If you want to see something creative, head on over to HR.com's virtual conference called VIEW, which is taking place today and tomorrow.

HR.com says this conference, which is very Second Life-ish, will have 40+ speakers, 1000+ attendees and 70+ vendors.

Right now I'm watching Carly Fiorina talk about leadership. Later presentations include:

Managing a Talent Pool for Succession Planning

The Federal E-Verify Program and Electronic I-9 Compliance

Quality of Hire

Creating Value Exchange in the Candidate Experience

...and a lot more. Creative stuff! And oh yah, it's free.

November '07 Issue of J.A.P.

The November 2007 issue of the Journal of Applied Psychology is full of interesting articles, including several relating to recruiting and assessment. Let's take a look:

First, a field study by Hebl et al. on pregnancy discrimination. Female confederates posed as job applicants or customers at retail stores, sometimes wearing a pregnancy prosthesis. As "pregnant" customers, they received more "benevolent" behavior (e.g., touching and over-friendliness), but as job applicants they received more hostile behavior (e.g., rudeness). The latter effect was particularly noticeable when confederates applied for stereotypically male jobs. This isn't a form of discrimination that gets as much play as others, but may be much more common than we think. My guess is a lot of people associate pregnancy with impending time off and don't focus as much on the competencies these women bring to the job.

Second, a study on faking. But wait, not faking on personality tests, faking during interviews. Levashina and Campion developed an interview faking behavior scale and then tested it with actual interviews. Guess what? Scores on the scale correlated with getting a second interview. (Looks like those classes you took on answering vaguely are going to pay off!) But wait, there's more. The authors also found that behavioral questions were more resistant to faking than situational questions (another reason to use 'em!), and follow-up questions INCREASED faking (another reason NOT to 'use em!). Other goodies in this article: over 90% of undergraduate job candidates fake during employment interviews (I assume that's just this sample), BUT, the percentage that were actually lying, or close to it, was less (28-75%).

Third, Brockner et al. provide research results that underline how important procedural fairness (justice) is. Three empirical studies demonstrated that employees judge organizations as being more responsible for negative outcomes when they experienced low procedural fairness. So when applicants or employees get bad news, they'll blame the organization even more if they feel the process used was unfair. Why do we care? Because perceptions of procedural fairness impact all kinds of things, including recruiting (e.g., how someone reacts to not getting a job) and the likelihood of filing a lawsuit (for, say, discrimination).

Fourth, Lievens, Reeve and Heggestad with a look at the impact of people re-taking cognitive ability tests. Using a sample of 941 candidates for medical school that took an admissions exam with a cognitive component, the authors found that retesting introduced both measurement and predictive bias: the retest scores appeared to be measuring memory rather than g, and predictive validity (of GPA) was eliminated. More evidence that re-testing effects are non-trivial. Pre-publication version here.

Last but definitely not least, one of my favorite topics--web-based recruitment. Allen, Mahto, & Otondo present results from 814 students searching real websites. When controlling for a student's image of the employer, job and organizational information correlated with their intention to pursue employment. When controlling for information search, a student's image of the employer was related to the intention to pursue employment, but familiarity with the employer was not. Finally, attitudes about recruitment source influenced attraction and partially mediated the effects of organizational information. What does all this mean? Don't throw your eggs into one basket--organizational image is important, but so is the specific information you have on your website about your organization and the specific job.

There's a lot of other good stuff in this volume, including articles on the financial impact of specific HRM practices, a meta-analysis of telecommuting impacts, engaging older workers, and daily mood.

EEOC Issues Fact Sheet on Employment Testing

On Monday, the U.S. Equal Employment Opportunity Commission (EEOC) issued a fact sheet on employment testing.

The fact sheet covers, at a high level, various topics including:

- types of tests

- related EEO laws (Title VII, ADA, ADEA) and burden-shifting scenarios

- the Uniform Guidelines on Employee Selection Procedures

- recent related EEOC litigation and settlements

- employer best practices

For those of you familiar with the legal context of employment testing, this isn't new information. But it is a nice quick summary of some of the major points, and could be very useful for someone not as familiar with this area.

For a more thorough treatment, I recommend the U.S. Department of Labor's guide for employers on testing and selection.

Winter '07 Personnel Psychology

Things are starting to heat up in the journal Personnel Psychology. The shot across the bow of personality testing that happened in the last issue of Personnel Psychology turns into a full-blown brawl in this issue. But first, let's not forget another article worth out attention...

First up, Berry, Sackett, and Landers revisit the issue of the correlation between interview and cognitive ability scores. Previous meta-analyses have found this value to be somewhere between .30 and .40. Using an updated data set, excluding samples in which interviewers likely had access to ability scores, and more accurately calculating range restriction, the authors calculate a corrected r of .29 based on the entire applicant pool. This correlation is even smaller when interview structure is high, when the interview is behavioral description rather than situational or composite, and job complexity is high. Why is this important? Because it impacts what other tests you might want to use--the authors point out that using their updated numbers they obtained a multiple correlation of .66 for a high structure interview combined with a cognitive ability test (using Schmidt & Hunters' methods and numbers). Pretty darn impressive.


Now that we have that under our belt, ready for the main event? As I said, in last issue Morgeson et al. came out quite strongly against the use of self-report personality tests in selection contexts--primarily because they claim the uncorrected criterion-related validity coefficients are so small. So it's not surprising that this edition contains two articles by personality researcher heavyweights defending their turf...

First, Tett & Christiansen raise several points; more than I have space for here. Some points include: considering conditions under which personality tests are used and validity coefficients aggregated; that there are occupational differences to consider; that coefficients found so far aren't as high as they could be if we used more sophisticated approaches like personality-oriented job analysis; and that coefficients increase when multiple trait measures are used. This sums their points up nicely: "Overall mean validity ignores situational specificity and can seriously underestimate validity possible under theoretically and professional prescribed conditions."

Second, Ones, Dilchert, Viswesvaran, and Judge come out swinging and make several arguments, including: conclusions should be based on corrected coefficients; coefficients are on par with other frequently used predictors, some of which are much more costly to develop (e.g., assessment centers, biodata); different combinations of Big 5 factors are optimal depending upon the occupation; and compound personality variables should be considered (e.g., integrity). Suggestions include developing more other-ratings instruments and investigating non-linear effects (hallelujah), dark side traits, and interactions. They sum up: "Any selection decision that does not take the key personality characteristics of job applicants into account would be deficient."

Not to be out-pulpited (yes, you can use that phrase), Morgeson et al. come back with a response to the above two articles, reiterating how correct they were the first time around. They state that much of what the authors of the above articles wrote was "tangential, if not irrelevant", that with respect to the ideas for increasing coefficients, "the cumulative data on these 'improvements' is not great", and that corrected Rs presented by Ones et al. aren't impressive when compared to other predictors. They point out some flaws of personality tests (applicants can find them confusing and offensive) but fail to mention that ability tests aren't everyone's favorite test either. They claim that job performance is the primary criterion we should be interested in (which IMHO is a bit short-sighted), and that corrections of coefficients are controversial.

So where are we? Honestly I think these fine folks are talking past each other in some respects. Some issues (e.g., adverse impact) don't even come up, while other issues (e.g., faking) are given way too much attention. It's difficult to compare the arguments side by side because each article is organized differently. It doesn't help that the people on both sides are some of the researchers with the most invested (and most to lose) by arguing their particular side.

I'm thinking what's needed here is an outside perspective. Here's my two cents: this isn't an easy issue. Criteria are never "objective." Job performance is not a singular construct. Job complexity has a huge impact on the appropriate selection device(s). And organizations are, frankly, not using cognitive or ability tests nearly as much as they are conducting interviews. So let's stop focusing on which type of test is "better" than the others. Frankly, that's cognitive laziness.

So is this just sound and fury, signifying nothing? No, because people are interested in personality testing. Hiring supervisors are convinced that it takes more than raw ability to do a job. We shouldn't ignore the issue. Instead we should be focusing on providing sound advice for practitioners and treating other researchers with respect and attention.

Should you use personality tests? I'll answer that question with more questions: what does the job analysis say? What does your applicant pool look like? What are your resources like? It's not something you want to use cookie-cutter, but not something you should write off completely.

Okay, I'm off my soap box. Last but not least there are some good book reviews in this issue. One is Bob Hogan's book (which I enjoyed immensely and actually finished which is rare for me), Personality and the Fate of Organizations, which the reviewer recommends; another is Alternative Validation Strategies, which the reviewer highly recommends; and the third for us is Recruiting, Interviewing, Selecting, & Orienting New Employees by Diane Arthur, which the reviewer...well...sort of recommends--for broad HR practitioners.

That's all folks!

Old-school competencies

Recently I went with family to the old Empire Mine in the gold country of California. It's one of the oldest, deepest, and richest mines in California. It produced 5.6 million ounces of gold before it closed in 1956.

In the display cases I happened to notice a brief description of the typical tasks and competencies required of a miner (a mini-job analysis if you will):











How would you have selected people for this job? Physical ability test? How about some type of personality test that measured flexibility and stress resistance? Might a realistic job preview have been a good idea?

Note it includes the traditional "what would happen if this job was done incorrectly..." question. This takes on increased meaning when you look at the shaft that the miners traveled down every day on their "way to work":











You may not be able to tell, but that sucker's pretty much straight down.

Suddenly my cubicle's looking a lot better.

Applied Workforce Planning: Air Traffic Controllers

There's plenty of disagreement over whether the aging of the U.S. workforce will indeed spark a wave of simultaneous retirements and thus a scramble to replace employees. Personally, I think it's going to depend on a lot of factors--the housing market, health care costs, and technological advancements to name a few.

But there are instances that demonstrate what it could look like if a significant portion of the U.S. workforce retires. One of those instances is the air traffic controllers who work for the Federal Aviation Administration (FAA) and were the subject of a recent piece on NPR. Let's look at the situation and the implications:

- In the summer of 1981, President Ronald Reagan replaced 12,000 striking air traffic controllers all at once. When you make such a large number of hires, in the same classification, with a lot of them probably being around the same age, in public sector (where folks often stick around for a long time), you better be thinking about workforce planning.

- Thirty percent more controllers retired last year than the Federal Aviation Administration (FAA) predicted. That's significant, and whether it's due to insufficient planning or not, it provides an example of the scope of the problem we're facing.

- Last year the FAA imposed a new labor contract on the controllers which lowered pay for new hires, froze pay for those with longevity, and placed new restrictions on the working environment. Not a great way to attract new candidates into a field where you need a high number of replacements in a short period of time.

- According to the union, employees feel stretched and burned out, which may lead to a serious accident. Not the type of publicity that demonstrates an outstanding employer brand.


Lessons? Like politics, all workforce planning is local. And yes, it's an inexact science. But sometimes the numbers just hit you over the head and demand attention. Do you have a situation like this in your organization? If so, what are you doing about it?

You don't know you

In a recent interview with Gallup Management Journal, Cornell psychologist David Dunning talked about why people aren't very good at judging themselves. Why is this important? Because it has a great deal to do with how we recruit and assess applicants.

A big part of the reason we're so bad at accurately judging ourselves is due to our self-serving bias--our tendency to take credit for successes but blame outside factors (other people, equipment) for our failures. This helps our ego out--if we were always blaming ourselves for failures and attributing success to outside factors, we wouldn't be very happy campers. But it has the downside of oftentimes blinding us to the real reason why things happen.

Dr. Dunning covers a wide range of topics in the interview, including gender differences, when overconfidence may be a good thing, employee training, providing feedback, and the serious implications of this phenomenon (e.g., think about doctors judging their skills as being better than they are).

As I said, this is important because a great deal of recruitment and selection is about self-assessment--a prime example is the growing movement toward online training and experience (T&E) questionnaires made easier with the spread of ATS products. Many of these questionnaires are chock full of questions that (no joke) aren't much different than: "How great are you at X?"

But it's not just about T&Es. People make judgments about themselves when deciding what jobs to apply for in the first place. They describe themselves in certain ways during job interviews (when the motivation to make yourself look good is even stronger).

What can we do about it? Simply put, verify, verify, verify. If someone claims to be the greatest Java programmer on the planet, make them show you. If they claim to be a great orator, make an oral presentation part of the hiring process. Then talk to folks that know their work to establish a history of competence. Don't take someone's word at face value because (a) they may be trying to snow you, but more subtly (b) they may not know themselves.

By the way, Dr. Dunning is co-author of one of my all-time favorite articles, "Flawed self-assessment: Implications for health, education, and the workplace" which describes how the inability of people to judge themselves accurately can result in very serious problems.

Last thing: if you're not already familiar with it, check out the fundamental attribution error, which is one of the other big things our brain is constantly doing. It has huge implications for how we judge others.

Rigorous assessment pays off

It's great when the mainstream press gets assessment right. It doesn't happen a lot, so I want to make sure to point out a good example.

Ellen Simon (AP) devoted a recent article to employers that, even in a tight labor market, put job applicants through the paces.

Some of my favorite bits from the article:

- Employers that recognize their employees are an integral part of their brand. If your employees are unhappy, not trained, or otherwise a bad fit, customers (and potential applicants) notice.

- This quote from Rackspace Managed Hosting CEO Lanham Napier: "We'd rather miss a good one than hire a bad one." Without getting into Type I versus Type II errors, let me just say that Mr. Napier demonstrates the wisdom of someone who's seen what a bad hire can (or can't) do. (Check out their refreshingly simple career portal)

- The fact that Rackspace interviews last ALL DAY. Yep, all day. In this age of "I only have 30 minutes for the interview", that's darn refreshing.

- The wonderful use of realistic job preview videos by Lindblad Expeditions that show employees cleaning toilets and washing dishes. Says Kris Thompson, VP of HR, "If they get on board and say, 'This is not what I expected,' then shame on us." Check out how their online preview video combines push with pull.

I don't agree with everything in the article--I'm not a big fan of the idea of secretly judging people on their waiting room behavior--but all in all some great examples here to recognize.

(by the way, the HBR article Simon cites, called "Fool vs. Jerk: Whom Would you Hire?" is here.

The changing requirement for war

The competencies required in the modern workplace change seemingly on a daily basis. Whereas computer skills were once a "nice to have" competency, now at least moderate computer proficiency is a "must have" for many (if not most) jobs. Whereas the ability to collaborate frequently with others may have been an infrequent requirement, now it is a prerequisite for many occupations.

Many times these changes are required to take advantage of new technologies and to keep up with competitors, and for the most part the transition works. On the other hand, sometimes we plan for one change in competencies when we should have gone another direction.

An excellent demonstration of this is the military. In this article from The Economist titled Armies of the future, the author describes the vision of former U.S. Defense Secretary Donald Rumsfeld:

The army's idea of its “future warrior” was a kind of cyborg, helmet stuffed with electronic wizardry and a computer display on his visor, all wirelessly linked to sensors, weapons and comrades. New clothing would have in-built heating and cooling. Information on the soldier's physical condition would be beamed to medics, and an artificial “exoskeleton” (a sort of personal brace) would strengthen his limbs.

At first, with initial successes in Afghanistan and Iraq, it seemed this vision would be validated. But as the nature of the warfare changed, so did soldier requirements. The current person in charge of the war in Iraq, General David Petraeus, has co-authored a new manual on counter-insurgency. According to the article:

Counter-insurgency, [the manual] says, is “armed social work”. It requires more brain than brawn, more patience than aggression. The model soldier should be less science-fiction Terminator and more intellectual for “the graduate level of war”, preferably a linguist, with a sense of history and anthropology.

This has huge implications for how the military will recruit, assess, and train soldiers, and we can argue about whether this shift could have been predicted or not, but there are some clear lessons here:

1 - Job analysis is important, and don't just do it once and put it in a file. Not that we needed any more evidence of this, but the example above vividly demonstrates how critical it is to carefully study a job (and keep studying it) to inform recruitment and assessment. Of course simply doing a job analysis doesn't guarantee success, as I'm sure there was a not-insignificant amount of thought that went into the "future warrior" idea.

2 - Look before you leap. Plan for how you will select people and train existing staff. The article doesn't go into this in depth, but one big implication for such a significant shift in competency requirements is what to do with existing staff. We've all been there--we implement a new software program, we require everyone to start writing or presenting more--and sometimes it works, sometimes it doesn't. Is our success due to the skill level of existing staff? Our careful planning? Or dumb luck?

When it comes to ensuring people are successful at what they do, fortunately we don't have to rely on luck. But we do have to devote time, resources, and careful attention to doing recruitment and assessment right. The consequences are important--whether they're saving lives or helping an organization be successful.

Jobfox members look upward

Jobfox is a job site I've posted about before that is making an attempt to more accurately match candidates with employers. The idea is to allow candidates to describe themselves in detail, including their work preferences, then have employers seek them (hence their motto, "be the hunted.")

Speaking of work preferences, one of the features the Jobfox offers is the ability for candidates to select up to five features of a job that they value the most--things like 401k matching, unstructured environment, and work/life balance.

Since Jobfox has all this information on people, they recently posted an analysis of results of over 6,000 registered job seekers. The press release focuses on the dearth of "green" factors people are looking for (e.g., looking for a company that is ecologically friendly), but to me the take home is about career advancement. Take a look at the top six desired job qualities:

Advancement opportunity (55%)
More leadership responsibility (41%)
Work/life balance (38%)
Leadership that's respected/admired (36%)
Sense of accomplishment (36%)
Higher salary (28%)

Notice that half of these, including the two most popular, are related to moving up in an organization.

The other result of note has to do with another kind of green (in the U.S. at least). Look at where salary is--down at #6. This suggests (and smart organizations know) there are ways of attracting and retaining talented folks simply by offering ways for people to take on increased responsibility and leadership opportunities, or restructuring the job (which might also help with that sense of accomplishment).