Thursday, May 27, 2010

Lewis case emphasizes need for valid tests


On Monday, May 24, the U.S. Supreme Court ruled in Lewis v. City of Chicago that plaintiffs filing an adverse impact discrimination claim under Title VII of the Civil Rights Act have 300 days to file from each time the test is used to fill a position--not just 300 days from when the test was administered.

For the most part, this mainly impacts employers that run an exam and use the results to fill positions for several years. In this case it was a firefighter entry-level exam, and my guess is it will mostly be public sector agencies and large employers that should pay particular attention to the ruling.

Why? Well for one it means more potential adverse impact lawsuits. If you were counting on being safe after 300 days from the exam, that is no longer the case. Second, it emphasizes the need to follow professional guidelines when developing an exam. Employers can successfully defend against an adverse impact case by showing that the selection practice is "job related for the position in question and consistent with business necessity..." (and that no alternatives with similar validity with less adverse impact were available). This means your exams need to be developed and interpreted by people who know what they're doing.

The City claimed that evidence related to an employer's business necessity defense might be unavailable by the time the lawsuit is brought--the court was not swayed. This means you'll want to hang on to your exam development records for at least a year beyond the last time you use the results to fill a position.

Another point worth noting: this case boils down to the validity of a cut score used by the City, which they themselves admitted wasn't supportable. Proper exam development and interpretation includes setting a job-related pass point based on subject matter input and/or statistical evidence that it is linked to job performance.

The Lewis ruling doesn't fundamentally change what we should be doing. It just emphasizes that we need to do it right.

Click here for a quick overview of the facts of the case.

Sunday, May 23, 2010

June 2010 IJSA

The summer journal season continues with the June 2010 issue of the International Journal of Selection and Assessment. Take a deep breath, there's a lot of stuff packed into this issue:

- Roth et al. provide evidence that women outperformed men on work sample exams that involved social skills, writing skills, or a broad array of KSAs. To the extent that an employer is trying to avoid discriminating against female applicants, this provides support for work sample usage.

- In a study of managers in Taiwan, Tsai et al. show that the most effective way an applicant can make up for a slip in an interview is to apologize (vs. attempting to justify or use an excuse).

- Jackson et al. strive to add some clarity on task-based assessment centers

- Blickle & Schnitzler provide evidence of the construct and criterion-related validity of the political skill inventory

- Colarelli et al. studied how racial prototypicality and affirmative action policies impact hiring decisions. Results of a resume review indicated more jobs were awarded to black candidates as racial prototypicality and affirmative action policy strength increased, but stronger AA policies decreased the percentage of minority hires attributed to higher qualifications.

- In my personal favorite article of the issue, Karl et al. found in a study of U.S. and German students that those low on conscientiousness (especially), agreeableness, and emotional stability were more likely to post "Facebook Faux Pas". This provides some support for employers who screen out applicants based on inappropriate social networking posts. I'll talk more about this in my upcoming webinar.

- Denis, et al. provide support for the NEO PI-R's ability to predict job performance in two French-Canadian samples.

- BilgiƧ and Acarlar report results of a study of Turkish students and perceptions of various selection instruments. Interviews were rated most highly and there were some differences in terms of privacy perceptions depending on the goal orientation of the student.

- Trying to figure out how to hire better direct support professionals (e.g., those providing long-term residential care or care to those with disabilities)? Robson, et al. describe the development of a composite predictor composed of various measures (e.g., agreeableness, numerical ability) that predicted performance, satisfaction, and turnover.

-
Ahmetoglu et al. provide support for using the Fundamental Interpersonal Relationship Orientations-Behaviour (FIRO-B) to predict leadership capability.

- Ispas et al. describe results of a study that showed support for a nonverbal cognitive ability measure (the GAMA) in predicting job performance in two samples.

- Last but not least, in another win for context-specific assessments, Pace & Brannick show how a measure of openness to experience tailored to specific work outpredicted the comparable general NEO PI-R scale. IMHO this is how personality measures will eventually become more prominent and accepted as pre-hire assessments.

Friday, May 21, 2010

IPAC conference to feature Campbell, McDaniel, Highhouse, and more

Those of you on the fence about attending the 2010 International Personnel Assessment Council (IPAC) conference on July 18-21 may be interested to know that a preliminary schedule has been released that reveals some great speakers and topics. For example:

- David Campbell's provocatively titled opening session, The Use of Picture Postcards for Exploring Diversity Issues Such as Bias and Prejudices, or "How Can We Keep Our Grandchildren From Going to War With Each Other?"

- Not to be outdone, Michael McDaniel kicks things off Tuesday morning with Abolish the Uniform Guidelines.

- Scott Highhouse closes things up Wednesday with A Critical Look at Holistic Assessment

- Great pre-conference workshops on everything from job analysis to fairness

- Wonderfully diverse concurrent sessions on topics such as public service motivation, leadership coaching, simulations, engagement, online testing, charging for exams, test transportability, cross-cultural personality assessment, measuring workforce gaps, adverse impact analysis, faking and lie detection, and succession planning. And that's just a sample!

Staying current on assessment through professional education is one of the commandments of our field. I hope you'll be joining your friends and colleagues in Newport Beach. Early bird registration ends June 1st.

Friday, May 14, 2010

Personnel Psychology, Summer 2010


The latest issue of Personnel Psychology (v63, #2) marks the beginning of summer journal season. Let's take a peek at some of what's inside:

Practice makes...better. John Hausknecht studied over 15,000 candidates who applied for supervisory positions (and 357 who repeated the process) over a 4-year period with a large organization in the service industry. The selection process included a personality test. He found that candidates that failed the first time around showed practice effects on dimension-level scores of .40 to .60. Candidates that passed the first time, but were taking the test again for other reasons, generally showed no difference in scores. More interestingly, on several subscales low scores the first time around were associated with practice effects that exceeded one standard deviation. A good reminder that personality inventories are susceptible to "faking", but certainly not a nail in their coffin as they still work quite well in many situations.

Another reason to structure your interviews.
As if you needed more convincing, McCarthy et al.'s study of nearly 20,000 applicants for a managerial-level position in a large organization found that the use of a structured interview resulted in zero main effects for applicant gender and race on interview performance. Similarly, there were no effects of applicant-interviewer similarity with respect to gender and race.

Users of the CRT-A take note. The conditional reasoning test of aggression (CRT-A) is used to detect individuals with a propensity for aggression. Previous studies have suggested the criterion-related validity of this test is around r=.44. In this study, by Berry et al., the authors meta-analyzed a large data set and found much lower values, in the .10-.16 range, that rose to .24-.26 when certain studies were excluded.

Assess your way into a job. Last but not least, Wanberg et al. describe the development of an inventory for job seekers called Getting Ready for your Next Job (YNJ, available here). The authors present results tying inventory components (e.g., job search intensity, Internet use) to subsequent employment outcomes.

Stay tuned, new issues of JAP, IJSA, and others should be out soon!

Wednesday, May 05, 2010

Grade your co-workers.com?


It was only a matter of time.

We know people like judging other people. Heck, we even like watching other people judge people. We also know people like interacting with websites rather than simply reading them. The natural result? Websites where you can judge other people.

Collective judgments are nothing new when it comes to restaurants, books, or even employers. But we're entering a new era where your online reputation may in large part be determined by other people. Assuming this trend takes off.

On one end of this spectrum, we have websites like Checkster, which is more of a reference checking or 360-degree feedback tool. It's what I would consider a "closed" system in its current iteration, because unless you're part of the process (applicant, employer, or reference-giver) you don't interact with it. The information remains relatively private and it's for a specific situation.

In the middle are sites liked LinkedIn, which allow you to "recommend" people. LinkedIn is a bit more open in that you can view someone's profile, but to see someone's recommendations, you need to be connected to them in some way, which is generally tricky for an employer unless they're already very connected. The other problem is the title--recommendations. This precludes other types of, shall we say, more constructive feedback.

On the far end of the spectrum is Unvarnished, which has recently gotten a lot of press. It's the most open system in that people's profiles are readily available (presumably; it's still in beta). Any unvarnished user can add a profile of a person to the site or comment on an already existing one. And it's all anonymous, although reviews can be rated and moderated. Finally, you can "claim" your profile and receive notification of new reviews, comment on ratings, and request reviews from specific people.

One of the big questions about this model is how accurate the information is. Are people just using this as an opportunity to get back at someone? Do they really know the person? To some these concerns are so overwhelming that they can't imagine using such a site. So it might be helpful for us to look at some recent research on a similar site, RateMyProfessors, which shares the open feel of Unvarnished.

You're probably familiar with RateMyProfessors. It's a simple way for students to provide feedback about their teachers. Teachers are rated on things like helpfulness and clarity and can provide comments as well. Those being rated can even provide responses.

Sounds like a way for failing students to rant about their professors, right? Well you might be surprised. In a new study published online, the authors looked at several hundred students and professors at the University of Wisconsin-Eau Claire. Here are some of their results:

1) Ratings were more frequently positive than negative.

2) "Popularity" (or lack thereof) of teacher was not correlated with frequency of feedback.

3) Students are not using the site to "rant" or "rave" as their primary motivation.

4) Those who posted were no different than those who hadn't in terms of GPA, year in school, or learning goal orientation. They were more likely to be male and their program was correlated with likelihood of feedback (e.g., those in the social sciences were more likely than those in the arts and humanities).

These results, if generalizable to other similar sites like Unvarnished, suggest that the results may be more accurate than we fear, and thus more useful. We know that peer reviews have at least moderate validity in terms of predicting performance. But there a still a lot of questions to be answered in terms of how the feedback is structured and how the information will be used by a potential employer.

So...might there be hope for crowdsourcing one's reputation? Or are we headed down a dangerous road? Would this make employers' lives easier--or just more confusing? Are defamation suits a possibility?

As an applicant yourself, here's something else to think about: would you rather your online reputation be determined by what an employer finds out about you while randomly surfing, or would you rather have a site where you can--at least partially--manage it?

Finally, consider this: If such a website became popular and filled with information about applicants...would you look someone up before hiring them?

Saturday, May 01, 2010

Webinar on Internet snooping


Okay, that's not the title. The real title is actually much lengthier but more accurate: "They posted what? Promises and pitfalls of using social networking and other Internet sites to research job candidates." Yours truly will be presenting this webinar--IPAC's first--on June 9th.

I believe Internet snooping is one of the elephants in the room when it comes to personnel selection--most people are doing it, but we don't talk about it. The way to deal with this is to get things out in the open and provide hiring supervisors with some informed guidance rather than pretend they're not doing it or deluding ourselves into thinking that blocking these websites at work takes care of it.

I'll be mainly focusing on two points: (1) why websites like Facebook and LinkedIn hold so much promise when it comes to gathering additional data on candidates, and (2) what the drawbacks are if you're going to do this. The latter includes things like ensuring authenticity, uncovering information you wish you hadn't, and finding the information in the first place.

It's free for IPAC members and $75 for non-members that includes a membership for the rest of the year. More details are here. Hope to "see" you there!

Tuesday, April 27, 2010

Assessments: More than meets the eye


So I quasi-randomly completed the 'personality quiz' over at the LA Times page. I did it just for fun, but it actually did something useful with my results--tailored my news. Although the label it gave me ("hot shot") is questionable, the stories it returned were based on the pictures I selected, things like politics, vacationing in Hawaii, and spending time with family. This led to a couple thoughts:

1) Why aren't more tests like this? Those of us on the professional side of testing often forget that there are tests that people actually enjoy taking. In fact people take "personality tests" all the time, through Facebook or on random websites. It's the kind of thing people pass around via e-mail. When was the last time you looked forward to taking an employment test?

2) I wonder what theory (if any) this is based on? It uses Imagini's VisualDNA technology, but I wasn't able to determine much from their website other than it took over three years to develop. Oh, and that apparently it's used by a number of sites, including match.com and hotels.com, for marketing purposes.

Taking this quiz also made me think not only about the "fun" side of testing but about alternate uses of assessment tools. These measures don't have to be used for selecting in and out. They can be used for many purposes, including some that are obvious (development) and some that perhaps aren't, like placement.

Using assessments for placement is something career counselors do all the time, but it's relatively rare for organizations. It shouldn't be. Imagine the value of putting some of your old-but-still-good assessments on the web and allowing people to take them, get feedback about their results, and receive some information that would allow them to self select in or out of various positions. It's a tool for insight, a realistic job preview, and an efficient way to populate the top of your selection funnel--all at the same time.

But wait, there's more. Imagine if you could populate your applicant tracking system with the results of said assessments. Imagine if, at the end of the assessment(s), the results strongly indicated the individual would be a good fit for a certain type of job. You could store their results for contacting in the future, provide them with additional recruiting material, lead them to relevant vacancies, and/or encourage them to apply.

Aside from some of the bleeding edge video game-type assessments, I haven't seen any selection tests that come close to fun (yes, I know we like to think that assessment centers are "fun" for applicants but we're fooling ourselves). And I don't recall seeing anyone using tests for placement in the way I described.

Have you?

Thursday, April 15, 2010

The perils of testing incumbents


Many of us plan or administer promotional tests on a daily basis. It's a normal way for the organization to figure out if people are ready for the next step--as journey person, lead, supervisor, manager, etc.

What isn't so usual is testing incumbents to determine if they can keep their jobs (drug testing aside). Amtrak, who is poised to take control of Metrolink, Southern California's commuter rail service, ran into fierce opposition recently when they announced that they will require train crews to take and pass two personality tests traditionally used only for screening applicants.

Why now? Amtrak is trying to prevent a repeat of the Chatsworth crash, where a Metrolink engineer crashed head on into a freight train, resulting in 25 dead (including the engineer) and 135 injured. Records revealed the engineer had a history of sending and receiving text messages hundred of times while operating trains (violating safety rules), including seconds before he ran a red led and crashed. Amtrak officials hope to identify "psychological issues", particularly those that manifest themselves during times of stress.

The tests in question are the Applicant Personality Inventory (API) and the Hogan Personality Inventory (HPI). The tests have been used since 2004 (API) and 2002 (HPI) to select among applicants for engineer and conductor positions, and the pass rate for the API is around 80%.

Union leaders have aggressively resisted the idea, claiming that the tests aren't valid or "relevant" measures of a trained and experienced employee's ability to safely operate trains. Instead, they claim the tests are a "witch hunt."

There are several things I find interesting about this situation:

1) Union leaders don't believe the tests are valid for incumbents, but have no problem with using them to select among potential hires. Assuming the personality aspects tested for by these measures are relatively stable, why would they be okay with testing applicants but not members? Methinks the issue is less validity and more membership.

2) If the vast majority of applicants, who are presumably a more heterogeneous group than incumbents, pass these exams, why would the union think that any incumbents would "fail" them? If it is indeed a witch hunt, what evidence do they believe management is relying upon to pre-judge certain individuals?

3) On a related note, given the high pass rates why would Metrolink/Amtrak think that any incumbents will "fail" these exams? (One wonders if the pass point would remain the same) This could be a situation where much political capital is expended with very little utility as a result of the assessments.

Incumbent testing is always a risky business. Even in the best of situations, those that fail or are passed over may harbor feelings of resentment, even anger. HR professionals must treat these selection situations with particular care and plan for how to communicate about the exam, before and after results are announced and used. But as this case demonstrates, when the tests in question are used to determine continued employment in one's current position--and the tests are personality inventories--tempers may flare particularly high.


Hat tip to my friend Warren Bobrow for this story.

Saturday, April 10, 2010

April 2010 TIP and latest EEO Insight

The April 2010 issue of The Industrial/Organizational Psychologist (TIP) is now available here. Let's check out some of what's inside:

- Several great, touching dedications to the late great Frank Landy. I found Rick Jacobs' eulogy particularly moving. My first real job out of graduate school was working for Frank doing research for expert witness testimony. Fabulous experience, amazing individual.

- Joel Wiesen proposes a novel approach to promotional selection in fire department settings.

- A great little article about moving from an I/O position to an HR generalist one. Very timely given all the changes HR departments are experiencing.

- The current status of legal protection for sexual minorities in the workplace

- A great article about the Bridgeport, CT case and why both the media and the city got things wrong.

- A fascinating breakdown of the major activities of I/O consultants v. internal practitioners v. academics. Check out Table 3.


In addition, there's a new issue of EEO Insight with some great content, and one article in particular I'd like to point out: A comprehensive look at how to successfully develop diversity initiatives and testing programs post-Ricci, including the essential elements of a Croson study (see Table 2) and the "strong basis in evidence" standard as applied to several practical examples (Table 3). It's a keeper and starts on page 27.

Some other articles to check out include:

- Strategies for defending and framing the issue of adverse impact in selection

- How to avoid adverse impact when choosing a test

Wednesday, April 07, 2010

Some blogs to follow + FB group

Here are three blogs you have missed:

Shaker Consulting Group (makers of the Virtual Job Tryout) has a new blog out written by my friend Joe Murphy. Recent posts covered the ERE conference, the importance of data-based decision making (can't agree with that enough!), and the recruiting value proposition.

Another is I/O at work, sponsored by HR Catalyst. They do a great job summarizing recent research across a wide spectrum of topics. Recent posts covered evidence-based management, the importance of positive feedback, and what job ads say about organizational culture.

Finally, those of you that subscribe to my shared items know I'm a fan of the blog from Recruitment Directory, an Australian consulting firm. Recent topics include checking out your job site on the iPad, website security, and the Australian HR Tech Report.

On a different note, if Facebook is a big part of your life, you may want to join the HR Tests fanpage. It's a great way to get your news in your feed, learn who else reads HR Tests, and have the ability to comment, without waiting for moderation, on any post.

Saturday, April 03, 2010

ClicFlic offers assessment innovation


A while back I posted about a creative use of technology that Vestas was using for onboarding. At the time I wrote about the potential I saw for the use of such technology for assessment, but actually creating these videos was a bit of a mystery from the customer side. Now I've come across a vendor that allows us to create these tools.

I don't post about specific products very often--usually I focus on research and best practices--but I have made occasional exceptions. When I see a product that I think has the potential to be innovative, highly effective, and highly valid, I want to share the wealth.

Such is the case with ClicFlic. In a nutshell, ClicFlic allows customers to create customized interactive web-based videos that can be used for things like situational judgment tests (SJTs). But we've seen that before, right? What I hadn't seen was the branching ability of ClicFlic.

Historically, video-based testing, whether Internet-enabled or not, presents all candidates with the same content. A situation is presented, and the candidate is provided with either several pre-determined responses or an open-ended response area. But much like traditional computerized adaptive testing (CAT), ClicFlic allows for the creation of branching videos. In other words, what the user sees in the next segment will vary depending on how they respond on the current one.

Although most of the examples you'll see on their website involve customer service or training applications, the technology is easily adaptable to assessment situations, as you can see from this example.

I had an opportunity to speak with Mike Russiello, President and CEO of ClicFlic (and co-founder of Brainbench) and he allowed me to peek "under the hood"--what I saw looked plug-and-play easy. The scripting branches are easy to generate, videos simple to upload, and you can quickly assign points to different responses. The videos are flash-based and you can easily generate the HTML to place it on a webpage.

Want to learn more? Check out the examples on their website--the demo on the front page will give you a good feel for the technology. Here are some others that will give you an idea of the possibilities. For assessment-specific usages, here you can select several different types of items with some characters you may recognize.

Questions? You can learn more about how the tools are built here. You may also run into Mike at SIOP if you have questions. Finally, they're also planning on an upcoming webcast through tmgov.org.

I hope this sparks some interest for you and maybe even some ideas about where this technology could be taken even further (RJPs anyone?).

Wednesday, March 31, 2010

This and that

I follow several journals, several of which aren't specifically devoted to recruitment and selection. But if you believe, as I do, that organizational structure and behavior have implications for what we usually talk about on this blog, I think you might find the following recently published articles interesting. I've also included a couple directly on point that you may have missed:

Got meetings? Turns out they're a key aspect of job satisfaction.

Thinking about work-life balancing measures? Consider the type of employee.

GLBT nondiscrimination policies may impact overall organizational performance.

Wrap your mind around this one: The ability to recognize opportunities may have a genetic component, similar to the personality aspect of openness to experience.

Are formal HR policies bad for morale? This study certainly suggests so. It also suggests that we need to "think small" when it comes to organizational units.

What makes someone "employable"? Willingness to change jobs--yes. Willingness to develop new competencies--not so much.

Interested in presenteeism (people coming to work sick)? Here's a good overview.

Maybe the New London police department wasn't so wacky. Turns out being overeducated negatively impacts job satisfaction--the good news is experience appears to moderate the relationship.

Bothered by the "criterion problem" in measuring the utility of assessments? This study won't make you feel any better, but it does help explain our challenge.

Want to do better on a test? Think positive.

Need more evidence that off-list checks are important? Check this out.

Saturday, March 20, 2010

March 2010 J.A.P.


The March 2010 issue of the Journal of Applied Psychology is out. Let's take a look:

Do women make better leaders? According to a study by Rosette and Tost, it varies with how success is attributed, the level of the position, perceptions of double-standards, and expectations. So the answer? A very solid "it depends."

Who should determine SJT scoring? Motowidlo and Beier suggest in their research study that situational judgment test (SJT) scoring keys based on input from subject matter experts (SMEs) contribute differentially to the prediction of job performance compared to keys based on general knowledge about trait effectiveness. What does this mean? That your ability to predict performance using SJTs depends in part on who is determining the scoring, and getting SME input may boost the effectiveness.

Do Americans work to live or live to work? Based on an analysis from Highhouse, et al., it's looking more and more like the former.

Need more evidence of the value of confirmatory testing? Naquin, et al. performed three experimental studies that demonstrated higher levels of lying when using email compared to pen and paper.

Do you like your leaders proactive? According to research conducted in China by Ning et al., you're not alone.

Finally, a slight correction to an article by Ilies et al. published last July on the relationship between personality and organizational citizenship behaviors.

Wednesday, March 17, 2010

2010 PTC -NC Conference

Last week I was fortunate to attend and present at the 2010 Personnel Testing Council of Northern California (PTC-NC) conference. Several of the presentation slides are now available.

My presentation was a legal update, primarily focusing on last year's big case, Ricci v. DeStefano. While I think the case received a fair share of its publicity simply because Sonia Sotomayor was one of the circuit court judges who ruled for the city, the case itself has some interesting implications for assessment. I gave my two cents last year after the decision.

Some of the points I made during the presentation included:

- Test validation standards as judged by the courts are generally very attainable. Following best practice (i.e., beginning with a thorough job analysis) is a recipe for a defensible process.

- Employers should spend the vast majority of their time before the assessment is given, figuring out what and how to test. Minimal time should be spent after the test making decisions about test usage--you should know that already.

- Employers, and assessment professionals, are expected to be familiar with and consider a wide range of testing mechanisms when planning a selection process. This includes non-cognitive assessments such as situational judgment tests, personality inventories, and biodata measures.

There were many great presentations; I always enjoy hearing what Wayne Cascio has to say, Dale Glaser has a way with statistics, and Deniz Ones and Stephan Dilchert's presentation on personality profiles of leaders was fascinating (they also happen to be very pleasant people to have lunch with). I plan on printing out slide 33 and placing it within reach--it does a great job of pointing out that criterion validity depends greatly on what you're trying to predict!

It's worth reviewing all the slides to get a flavor of what was discussed; there will likely be other presentations added over time.


On a side note, I'd like to acknowledge my readers at Baruch College -- thanks for reading!

Sunday, March 14, 2010

Hiring Right Presentation


Late last year I did a webinar through BCGi titled "Hiring right: How to hire the right person for the job" and the slides are now available.

For many of you it will be review, but I tried to cover a variety of topics relevant for newer professionals as well as hiring supervisors, including:

- accuracy of our perceptions regarding hiring

- reviewing applications and resumes

- types of assessment

- interview questions

- best practices

You can see the PDF version here; content begins on page 5.

Wednesday, March 03, 2010

Personnel Psychology, Spring 2010: SJTs, affect, and job offer timing


The Spring 2010 (v.63, #1) issue of Personnel Psychology is out. Let's look at the highlights:

First out of the gate, a great meta-analysis for anyone interested in situational judgment tests (SJTs; and who isn't?). Christian, et al. looked at 84 studies and found some pretty interesting things:

1) SJTs reported in the literature have been used to measure a variety of things, including leadership skills (37%), some type of composite (33%), interpersonal skills (12.5%), personality tendencies (9.6%), teamwork skills (4.4%) and job knowledge/skills (3%).

2) Criterion-related validity depends--as you might expect--on the match between predictor and performance measure. Conscientiousness measures, for example, predicted task performance much better than managerial performance (rho=.39 and .06 respectively). The highest correlations (albeit based on relatively small samples) were for teamwork skills and personality composites predicting task performance (.50 and .45 respectively).

3) Video-based SJTs tended to have stronger criterion-related validity values compared to paper-based measures. This was particularly true when measuring interpersonal skills (.47 compared to .27).

Second, a small but interesting study by Johnson, et al. on the relationship between trait affect (i.e., being generally disposed to feeling positive or negative emotions) and job performance. Results from 120 matched employee-supervisor pairs from a variety of jobs using both explicit (survey) and implicit (word fragment completion) measures of affect found substantial correlations, particularly between positive affect and performance (in the .50 range), and particularly when using implicit measures.

Something to add to a selection battery, perhaps? Could be perceived negatively by applicants, however, and I can see some questions being raised about the link to medical issues. But the same types of concerns were originally leveled at personality tests and were mitigated by creating measures specifically tied to work behavior. Definitely an area for more research.

Third, check out this study by Becker, et al. on the impact that job offer timing has on acceptance, performance and turnover. The authors found (using data from a Fortune 500 engineering technology company) that for both student and experienced samples, faster offers were associated with higher acceptance rates. Specifically, for experienced candidates, the difference between 2 weeks and 3 weeks taken to make the offer was substantial, whereas for the students 3 weeks versus 4 weeks was important. But, no differences were found in terms of either performance ratings or turnover among employees hired through different offer speeds.

Implication? The study suggests that offer time does impact the likelihood that the offer will be accepted, but viewed broadly this may not have long-term impacts in terms of how employees do on the job. Maybe in cases of good candidate-employer fit, candidates are willing to wait.

Last but not least are the book reviews. Two books are particularly relevant for us, The Structured Interview (Pettersen & Durivage) and Outliers (Gladwell). The first is received very positively and sounds like a great source for anyone wanting more details about the support for and use of structured interviews. The latter is "well worth [a] few evenings" but requires you to overlook the lack of evidence and convenient inferences.

Final notes: those of you interested in multisource performance ratings should check out Hoffman, et al.'s article, which reinforces the impact of having raters from different levels. Chuang and Liao's article also includes a useful measure of a high-performance work system.

Tuesday, March 02, 2010

IPAC Conference + Innovations Award


What are you doing July 18-21? I assume if you enjoy good weather, good company, and--most importantly--great information on state-of-the-art selection practices, you'll be joining me at IPAC's annual conference in beautiful Newport Beach, California.

If you're not only going but have something to present, by all means respond to the call for proposals. It could be a workshop, panel discussion, symposia--pretty much any format you can think of. Don't wait too long, the deadline is this Friday, March 5.

And speaking of the conference, IPAC has announced that nominations for the Innovations in Assessment Award are being accepted from 5/17-6/18. The winner receives not only formal recognition (and bragging rights), but a free pass to the conference.

IPAC's a great group, full of people that are extremely knowledgeable and passionate about using the best selection practices to get organizations the talent they need. Plus, it's the only international (or national) organization I know of devoted exclusively to the topic.

Hope to see you there!

Friday, February 26, 2010

Book review: Strategy-Driven Talent Management


A thought-provoking collection of essays and ideas; but it won't solve all our problems.

The value of a book lies as much with the reader as it does with the content. A book about advanced programming does little good to the person who has problems turning a computer on. A collection of cooking recipes is largely useless to someone who exclusively uses a microwave.

The same is true about business and HR books. Depending upon who you are and where you're at in life, some books may help you, some may be beyond your reach. Such is the case with SIOP's latest entry into its Professional Practice series, Strategy-Driven Talent Management: A Leadership Imperative, edited by Bob Silzer and Ben Dowell.

The book (tome, actually, at nearly 900 pages) is full of thought-provoking pieces from a variety of authors, including some familiar faces such as John Boudreau and Allan Church. There are academics present, but the majority of authors are practitioners in private sector organizations, such as Aon, Ingersoll Rand, HP, Sara Lee, Merck, and Bristol-Myers Squibb.

The book is roughly broken up by major topic area, although the distinction can be hard to maintain. There are chapters on recruitment, executive onboarding, engagement, measurement, and global issues. There's even a 40-page annotated bibliography. But the editors do an admirable job of keeping the topics all related to the broad field of talent management (TM), which they define as, "an integrated set of processes, programs, and cultural norms in an organization designed and implemented to attract, develop, deploy, and retain talent to achieve strategic objectives and meet future business needs" (p. 18).

The book is described as a "comprehensive [collection of] state-of-the-art ideas, best practices, and guidance." It shines on the first two but failed me on the last, although not for lack of trying. The problem is the book is so long, full of so many ideas and case studies, that it's very easy to get lost and not come away with any clear guidance based on the consensus of authors. To some extent this is endemic in any collection of works by separate authors, but it's clearly a collection of "what you might do" rather than a solid prescription for "how to", although some authors do a better job than others.

Another problem is that many authors seem to presume that current TM practices are sub-optimized because they aren't linked to business strategy and results. This may be true if the process is based on non-validated assumptions, but as long as there is a link between job success and specific practices, we're already there. We just haven't made a particularly good link between job success and organizational success, which may explain the attraction to concepts like competencies (mentioned many times in the book).

But my main problem, and this goes for the book as well as the field, is that it treats the concept of talent management as a logical process to be managed. Somewhere in the transition from HR to TM, we lost the H--human. Talent management (and HR) is messy because it involves people. It's political. It changes every day. And you're dealing with emotions, not lines of code. The real challenge--which is discussed but to my mind not driven home--is how to get the talent mindset into the organizational DNA.

There is value to thinking broadly and philosophically about the topic. It helps us plan. But what people really need are concrete suggestions for establishing a self-sustaining high-performance system. In order to do this, we must address the fundamentals (the basic needs of Maslow's hierarchy, if you will), such as:

- HR must learn "the business" and stay close to their customers
- Supervisors must be selected and trained with their talent management role at the forefront
- Success in HR must be defined and measured. It must be communicated, understood, and valued
- Sustained attention to HR success and significant resources must be expended by both HR and line managers

The book does a passable job of presenting these, but you may have to dig for them. The bigger problem is that there seems to be an assumption that what keeps organizations from having a top-notch TM system is a lack of understanding, either of the organizational strategy or best practices in TM, rather than the very real daily troubles that organizations experience, such as:

- Supervisors that hire people they know/like rather than the most qualified person
- People placed into HR with little or no background, interest, or passion for it
- Insufficient resources devoted to TM/HR
- HR managers who are just that--managers--rather than real HR leaders (Avedon and Scholes present a great assessment in Chapter 2 that helps separate these)

Until organizations have these types of "minor"--but real--flaws ironed out, all the charts and good intentions in the world will have very little impact.

Finally, I was also disappointed that there wasn't more in here about evidence-based TM and HR (which may say more about the field than the authors/editors, who acknowledge this lack in Chapter 22). The field desperately needs more research to tie the hard science of assessment with the more anecdotal/consultant practices such as recruiting, retention, and performance management. This will require significantly more research using methods beyond surveys in order to show what works and what doesn't. There are some ties to good research in here, but the hole is significant.


To summarize, the book contains a lot to like, particularly for individuals already schooled in this area looking to optimize their shop, or for graduate students seeking to understand the big picture. But for most HR practitioners (and, I would expect, executives), this book is akin to a collection of recipes for advanced Italian cooking--fabulous for those used to making their own pasta, but beyond the reach of those struggling to make their own sauce.

Thursday, February 25, 2010

Webinar on 21st Century Assessment


Went to a pretty darn good webinar yesterday put on by HCI and featuring Ken Lahti (PreVisor) and Charles Handler (Rocket-Hire). The topic was 21st century assessment.

Some of the topics covered included:

- increased functionality and usability of testing platforms

- increased sophistication of security methods

- off-the-shelf tests and "I/O psychologists in a box"

- integrating assessment with your overall talent strategy

And my two favorites:

- advanced simulations (such as those using video game technology)

- candidate data that follows them

The webinar is going to be re-broadcast several times today and tomorrow, if you have a chance check it out. You can also see a copy of the slides for free if you're an HCI member (which is free).

Free, short, and full of information--that's my kind of training.

Saturday, February 20, 2010

Recruitment v. Assessment, Round 1: Fight!


I'm sure some of you are avid readers of ERE (Electronic Recruiters Exchange), but for those of you that aren't (and don't receive my shared items), there's a lively discussion going on over there regarding the practical value of recruitment versus assessment practices.

It started with Wendell Williams' first post on how to identify a bad test (the second part is also worth reading). The comments begin relatively benignly, debating the strength of various predictors of performance (e.g., P-O fit versus behavioral interviews), but turns into, let's say, a lively debate that includes a discussion of the Gallup 12, the limitation of assessments, the Uniform Guidelines, and a lot more. The most heated exchange occurs between Wendell and Lou Adler, where accusations and sarcasm fly.

Speaking of Lou, he continues to advocate his perspective with his next post on whether increasing interview accuracy increases quality of hire (yes, he's suggesting that's an open question). While the comments following are fewer in number, the debate continues regarding the value of assessment and the evidence used to support it (e.g., Schmidt & Hunter's 1998 piece).

Who said HR is boring?