Monday, August 06, 2007

2007 Academy of Management Conference

There have been some news stories about one of the presentations at this year's Academy of Management (AOM) conference--about an online survey where a majority of respondents said that bad bosses either get promoted or have nothing happen to them. But there's a heck of a LOT of other good stuff at this year's conference. So take a deep breath and let's take a look...

First up, a whole set of presentations devoted to selection, including:

- Hiring for Retention and Performance
- Work Sample Test Ethnic Group Differences in Personnel Selection: A Meta-analysis
- Stigmatizing Effects of Race-Based Preferential Selection
- Longitudinal Changes in Testing Applicants and Labor Productivity Growth



Next, a session devoted to recruitment and selection, including:

- The Role of Sociolinguistic Cues in the Evaluation of Job Candidates
- Recruitment as Information Search: The Role of Need for Cognition in Employee Application Decisions
- A House Divided: Cooperative and Competitive Recruitment in Vital Industries
- The Practice of Sense-Making and Repair during Recruitment Interviews
- Overqualified Employees: Too Good to Hire or Too Good to Be True?



Next up, a session devoted to recruitment. Included topics:

- Customizing Web-Based Recruiting: Theoretical Development and Empirical Examination
- Network-based Recruiting and Applicant Attraction: Perspective from Employer and Applicants
- Fancy Job Titles: Effects on Applicants' Job Perceptions and Intentions to Apply
- Recruitment and National Culture: A Value-Based Model of Recruitment



Next, a set devoted to person-organization (P-O) fit, including:

- Going Beyond Current Conceptualizations of P-E Fit and Presenting a Status Report on the Literature
- Outcomes of Multidimensional Misfit: An Empirical Test of a Theoretical Model
- FIT: Scale Development and Initial Validation of a New Measure
-
Considering the Contextualized Person: A Person-In-Content Approach to Goal Commitment


Next, a set on predictors of individual performance, including:

- An Examination of Ability-based Emotional Intelligence and Job Performance
- Predicting NFL Performance: The Role of Can-do and Will-do Factors
- A Fresh Perspective on Extraversion and Automobile Sales Success
- Auditor Effectiveness and Efficiency in Workpaper Review: The Impact of Regulatory Focus



Last but not least, one of my favorite topics, how organizations and individuals perceive selection. Topics include:

- Understanding Job Applicant Reactions: Test of Applicant Attribution Reaction Theory
- Effects of Ingratiation and Similarity on Judgments of P-O Fit, Hiring Recommendations and Job Offer
- The Effects of Resume Contents on Hiring Recommendations: The Roles of Recruiter Fit Perceptions
- Organization Personality Perceptions and Attraction: The Role of PO Fit and Recruitment Information



This is just a sample of what the conference has to offer; if you went, or otherwise know of other presentations we should know about, please share with us.

And no, most of the presentations aren't available on-line but the presenters' e-mail addresses are provided and most folks are willing to share.

Thursday, August 02, 2007

Is USAJobs enough?

Check out this article that came out recently on GovernmentExecutive.com. It's about how the federal government may need to branch out and start using other advertising venues besides USAJobs.gov, which it relies on heavily.

Some individuals quoted in the article, which happens to include a manager at CareerBuilder, point out that:

- Opportunities are not automatically posted on other career sites, like CareerBuilder, Monster, and HotJobs.

- Job openings are not "typically" searchable through search engines like Google. (Although look what happens when I search for an engineering job with the federal government).

- You can't expect people to automatically look for jobs on USAjobs.

The Office of Personnel Management (OPM), the fed's HR shop, fires back:

- USAJobs gets 8 million hits a month. This compares to CareerBuilder's 1.2 million searches a month for government jobs.

- USAJobs is well known and marketing efforts have been ramped up (e.g., last year's television commercials, which unfortunately didn't work with my version of Firefox).

So who wins the argument? I don't think the feds need to panic just yet. But it can't hurt them to investigate other posting opportunities, particularly given how much traffic the heavy hitters like Monster and CareerBuilder get compared to USAJobs:

By the way, don't overlook the comments on that page; in some ways they are more telling than the article. Readers point out that the application process is overly complicated--to the point that one of the readers makes his/her living guiding people through the process (reminds me of a guy that does the same thing for the State of California). My bet is the application process is equally, if not more, important than how the feds are marketing their opportunities.

I would also be willing to bet that it isn't just the feds that have this issue. As more organizations implement automated application and screening programs, they risk falling in love with the technology at the expense of the user experience. I may love the look of your job, but if it takes me 2 hours to apply, well...I may just look elsewhere.

Tuesday, July 31, 2007

B-school requires PPT slides for admission

So apparently Chicago's Graduate School of Business is going to require four pages of PowerPoint-like slides as part of its admission process this fall.

According to school reps, this will allow students to "show off a creative side that might not reveal itself in test scores, recommendations and even essays." Another rationale given by the school is that students will have to master this type of software before entering the business world.

One problem I see here is the vast majority of applicants will already know PowerPoint--if you get through high school and college without using it, I'm betting you're the rare applicant.

The larger problem here is the same problem employers face with supplemental questionnaires and work samples--namely, who did it? In high-stakes situations like school admissions and job applications, people are known to take, shall we say, less than ethical routes to increase their chances.

The benefit of something like GPA or the GMAT is identity verification--you can be virtually assured (as long as you can validate the numbers) that the person who's applying is the one that took that test.

With things like previous work samples, resumes, and now this PowerPoint idea, you have no idea who actually created the product. So you make an admissions or hiring decision based on an assumption. Do you validate that they actually created these documents? Probably not. Even if you wanted to, how would you do it?

It might not even matter, since this may be more of a way to add excitement to application reviews and to simply get more applicants, which the school acknowledges. There seems to be a trend among organizations to implement projects that aren't so much concerned with valid predictions of performance but with simply attracting attention. This will likely get even more blatant as organizations struggle to keep their staffing levels up in the coming years.

But we should keep this in mind: gimmicks may attract some applicants, but do they turn others away? What about highly qualified individuals who think, "Well that's silly." That's why the best solutions will pique interest while being as close as possible to the actual job (or school) environment. How about asking applicants to give a presentation as part of their interview--now that's a skill they'll need. Plus, absent any Mission Impossible-style disguises, you can be pretty sure the person in front of you is who they claim to be.

Monday, July 30, 2007

Webinar on Assessment Best Practices

On July 25, 2007, Dr. Charles Handler of rocket-hire.com gave a very good overview of best practices in a webinar titled Screening and Assessment - Best Practices for Ensuring a Productive Future.

Some of the topics he covered included:

- Different types of assessment
- Planning for assessment
- Validity and prediction
- Screening out vs. screening in

You can view a pdf of the slides here, and listen to an mp3 of his presentation here.

Wednesday, July 25, 2007

Tuesday, July 24, 2007

IPMAAC Presentations + Cheaper Membership

I posted earlier about presentations from the 2007 IPMAAC conference going up online. Well now there's a whole gaggle of 'em and there's some really great content. Check out these sample titles (PDF):

Applicant Reactions to Online Assessments

Succession Planning and Talent Management


2007 Legal Update

Potholes on the Road to Internet Applicant Compliance

Measuring Complex Reasoning


Tips on Writing an Expert Witness Report

And that's just the beginning. For all the goodies, check out the full list.

But wait, there's more...

In addition, IPMAAC recently enacted a change to its membership categories & fees. You can now become an IPMAAC member for only $75! Talk about cheap. $75 pays for the difference in conference fees between a member and a non-member! And you get all this to boot. Plus, you're "affiliated" with IPMA-HR, which means you get the awesome weekly HR newsletter and discounts on all sorts of IPMA-HR stuff (that's a technical term). And you DON'T have to work in the public sector to join.

There really aren't that many professional organizations associated with assessment. There's SIOP, but they're about a lot more than just staffing. There are local groups. But when it comes to national or international groups, IPMAAC is it. Which is a good thing, because it's a great group of people (not that I'm biased or anything).

Saturday, July 21, 2007

Legos: They're not just for Google anymore

So apparently Legos (or "Lego bricks") are enjoying quite the popularity among corporate recruiters these days.

Not only did Google use them at Google Games (and apparently employees enjoy them as well), PricewaterhouseCoopers (PwC) asks candidates to create a tower with Legos, according to this Economist article.

So what exactly are candidates doing? PwC asks candidates to create the tallest, sturdiest structure they can using the fewest number of "bricks." Google asked candidates to build the strongest bridges they could.

Is this a valid form of assessment? A "professional Lego consultant" in Buenos Aires stated that, "Lego workshops are effective because child-like play is a form of instinctive behaviour not regulated by conscious thought. " There's even a website devoted to Lego's efforts in this area--Serious Play.

So my question is: Do most of us do work that is "not regulated by conscious thought"? Perhaps sometimes, say in emergencies. But the vast majority of time we're pretty darn deliberate in our actions. The only situation I can see where this might be predictive of actual job performance would be for jobs like bridge engineer or architect. But...computer programmer? If I wanted to know how creative a programmer is, I'd ask him/her to solve a difficult coding problem.

Does this even matter? Perhaps not (unless they're sued). As one of the candidates states, correctly I think, "It was as much advertising as a way of trying to get recruits." So in this day and age of "talent wars", this may be just another branding technique.

Will it be successful? Probably depends on how much the candidate likes to play with blocks.

This post is dedicated to my Grandpa Ben, who had a great sense of humor. And probably would have thought using Legos in this way is a bit silly :)

Thursday, July 19, 2007

New issue of Journal of Applied Psychology (v.92, #4)

Guess how many articles are in the most recent Journal of Applied Psychology. Go ahead, take a gander.

10? 15?

Try 23. I mean....that's just showing off.

So what's in there about recruitment & assessment? Believe it or not, only two articles. Let's take a look at 'em.

First up, a study by Klehe and Anderson looked at typical versus maximum performance (the subject of the most recent issue of Human Performance) during an Internet search task. Data from 138 participants indicated that motivation to perform well (measured by direction, level, and persistence of effort) rose when people were trying to do their best (maximum performance). But the correlation between motivation and performance diminished under this condition, while the relationship between ability (measured by declarative knowledge and procedural skills) and performance increased.

What the heck does this mean? If you're trying to predict the MAXIMUM someone can do, you're better off using knowledge-based and procedure-based tests. If, on the other hand, you want to know how well they'll perform ON AVERAGE, check out tests that target things like personality, interests, etc.

Second, Lievens and Sackett investigated various aspects of situational judgment tests (SJTs). The authors were looking at factors that could increase reliability when you're creating alternate forms of the same SJT. Using a fairly large sample (3,361) in a "high-stakes context", they found that even small changes in the context of the question resulted in lower consistency between versions. On the other hand, being more stringent in developing alternate forms proved to be of value.

What the heck does this mean? If you're developing alternate forms of SJTs (say, because you give the test a lot and you don't want people seeing the same items over and over) this study suggests you don't get too creative in changing the situations you're asking about.

As usual, the very generous Dr. Lievens has made this article available here. Just make sure to follow fair use standards, folks.

Monday, July 16, 2007

Finally, an assessment gameshow

Okay, so it's not "guess the criterion-related validity", but it's about as close as we're going to get to a game show focused on assessment (The Apprentice notwithstanding).

The show is called "Without Prejudice" and it premiers July 17th on the Game Show Network (GSN). The concept is that a diverse panel made up of "ordinary members of the public" will be judging a similarly diverse group of people and determining who should be given the $25,000 prize.

So how is this like assessment, you say? Well the judges have to decide who they like the most or hate the least and use their judgment to determine who to award the prize to, based on seeing video clips of the contestants and information about their background. What does this sound like? Your average job interview!

What does it look like? In the premier, the first task the panel must do is decide who among the five contestants should be denied the money based on a very quick (about 5 seconds) introduction by the person. The panel focuses heavily on appearance rather than what was said, including making judgments about how wealthy the person is, their age, and their "vibe."

In an interesting twist, the host talks to the people that were "eliminated" about how they felt. (Ever asked a denied job applicant how they were feeling? Could be informative.)

Is it overly dramatic? Absolutely. Will it last? Probably not. Does it give us a vivid example of how quickly impressions are made, and on what basis? Yep.

There's a sneak peek of the premier available on the website. There is also, to their credit, links to information about prejudice, including two questionnaires you can take to probe your own beliefs.

Friday, July 13, 2007

New issue of Human Performance: Typical v. maximum performance

There's a new issue of Human Performance out (Volume 20, Issue 3) and it's devoted to a very worth topic--typical versus maximum performance.

What is the distinction, you say? Well it's pretty much what it sounds like. From Smith-Jentsch's article in this issue:

"Typical performance in its purest sense reflects what a person "will do" on the job over a sustained period of time, when they are unaware that their performance is being evaluated. By contrast, maximum performance reflects what a person "can do" when they are explicitly aware that they are being evaluated, accept instructions to maximize effort, and are required to perform for a short-enough period of time that their attention remains focused on the task."

The recent interest in this area stems largely from Sacket, Zedeck, & Fogli's 1988 article in Journal of Applied Psychology. Previous research suggested that measures of ability (e.g., cognitive ability tests) more accurately predict maximum performance whereas non-ability measures (e..g, personality tests) are correlated more with typical performance. This of course has implications for who we recruit and how we assess: Are we trying to predict what people can do or will do? The answer, I think, depends on the job--for aircraft pilot or police officer, you want to know what people can do when they're exerting maximum effort. For customer service representatives, you may be more interested in their day-to-day performance.

This topic is mentioned often in I/O textbooks but (as the authors point out) hasn't been researched nearly enough. The authors of this volume attempt to remedy that in some part. Let's look at the articles in the recruitment/assessment area:

First, Kimberly Smith-Jentsch opens with a study of transparency of instructions in a simulation. Analyzing data from two samples of undergraduates, the results validate previous findings: Making assessment dimensions transparent (i.e. telling candidates what they're being measured on) allows for better measurement of the abilities necessary for maximum performance, while not making this information transparent appears to result in better measurement of traits that motivate typical performance. So if the question is, "Do we tell people what we're measuring?" the answer is: It depends on what your goal is!

Next, Marcus, Goffin, Johnston, and Rothstein tackle the personality-cognitive ability test issue with a sample of candidates for managerial positions in a large Canadian forestry products organization. The results underline how important it is to recognize that "personality" (measured here by the 16PF and PRF) has several components. While cognitive ability scores (measured by the EAS) consistently outperformed personality scores in predicting maximum performance, measures of extraversion, conscientiousness, dominance, and rule-consciousness substantially outperformed cognitive ability when predicting typical performance.

Third
, Ones and Viswevaran investigate whether integrity tests can predict maximal, in addition to typical, performance. The answer? Yes--at least with this sample of 110 applicants to skilled manufacturing jobs. Integrity test scores (measured using the Personnel Reaction Blank) correlated .27 with maximal performance (measured, as is typical, with a work sample test). The caveat here, IMHO, is that job knowledge scores correlated .36 with maximal performance. So yes, integrity test scores (a "non-ability" test) can predict maximal performance, but perhaps still not as well as cognitively-loaded tests.

Last but not least, Witt & Spitzmuller look at the relationship between cognitive ability and perceived organizational support (POS) on typical and maximum performance. Results from two samples (programmers and cash vault employees) reinforce the other results we've seen: Cognitive ability (measured by the Wonderlic Personnel Test), was correlated with maximum performance but not typical performance, while POS was related to two out of three measures of typical performance but not with maximum performance.

Overall, the results reported here support previous findings: maximum performance is predicted well by ability tests while typical performance has stronger correlations with non-ability tests. But Ones & Viswevaran are correct when they state (about their own study): "Future research is needed with larger samples, different jobs, and different organizations to test the generalizability of the findings." Let's hope these articles motivate others to follow their lead.

Wednesday, July 11, 2007

FBI settles sex discrimination case for $11M

The Federal Bureau of Investigation (FBI) has settled a class action sex discrimination lawsuit brought on behalf of current and former female employees who were prohibited from applying for numerous GS-14 and GS-15 promotions. The case was titled Boord v. Gonzales.

These higher-paying jobs were restricted to individuals that had experience as a Special Agent and included a range of position titles, including EAP counselor, laboratory analyst, and lawyer. For many of these, according to the plaintiffs, having Special Agent experience was not job-related and consistent with business necessity, as required under Title VII of the Civil Rights Act of 1964.

The total cost: Approximately $11 million

Dr. Kathleen Lundquist, well known in I/O circles, served as a neutral expert to review positions that required Special Agent experience. The request for position review form can be found here.

Lesson? Make sure your minimum qualifications are valid. They are open to scrutiny--just like every other part of the screening process.

More details can be found in the settlement agreement.

Monday, July 09, 2007

TalentSpring takes on peer ratings

A while back I mentioned RecruitmentRevolution, a UK site focused on temporary employment that allows previous employers to input reference scores for use by future employers. Creative idea, if ya ask me. I've often wondered if someday there will be a general database of verified work history that employers could easily check.

Now along comes TalentSpring with a similar idea. This time it's not previous employers, it's peers. TalentSpring uses something it calls a "Merit Score." From the website:

"TalentSpring creates accurate merit scores by using the votes from candidates. Advanced mathematics are used to detect inaccurate votes and remove them while still accurately ranking candidates. The top resume in an industry receives a merit score of 2,000 and the most entry level candidate receives a merit score of 1,000."

How does it work?

"The voting process used to generate the Merit Score rankings is very simple. Voters are shown a series of resume pairs. With each pair the voter is asked which of the two candidates is most likely to be brought into an interview for the typical job opening in this job category. It is that simple - is Candidate A or Candidate B better in this job category. There is no worrying about previous pairs or what resumes are going to show up next. Each pair is considered in isolation."

And who's voting?

"Your resume is voted on by other people seeking to be ranked in the same category you are. Just as you are voting on other candidates in the same job category you are in. Since TalentSpring offers quite a few job categories to choose from, on occasion you may be voting on (and be voted on by) candidates in related job categories. For example, a C++ programming candidate might end up voting on Java programmers."

What about accuracy?

"We know when people are voting outside the "normal" range and remove these votes from the ranking calculations. We think that the ability to accurately vote is a skill that recruiters are interested in because it reflects both your understanding of the position you are interested in and your attention to detail. That is why we calculate and post your voting score as part of your Candidate Overview."


So what do you think? I was disappointed at who's doing the ranking--I assumed by "peers" they meant one's actual co-workers. Now that would be interesting, given that these type of peer ratings are at least partially trustworthy. I wonder how accurate ratings of competing job hunters will be. With no control over subject matter expertise, this relies solely on (what I assume is) statistical abnormalities. Not particularly encouraging. In addition, selecting one person over another for a general job category may prove to be an impossible task, as even jobs in a single category can vary substantially in terms of competencies/KSAOs required.

BUT, that said, it is encouraging to see steps taken in a different direction. If we could just combine some of these approaches, we may be making our way slowly toward a database that employers could feel good about. Of course that means a whole other discussion on rating formats...

Hat tip.

Friday, July 06, 2007

EEOC Releases Updated ADEA Regulations

Today the U.S. Equal Employment Opportunity Commission (EEOC) issued final revised regulations on the Age Discrimination in Employment Act (ADEA) that take into account the Supreme Court's 2004 decision in General Dynamics v. Cline. In Cline the court determined that employers could lawfully give preference to an older worker over a younger worker in situations where both were covered by the Act (i.e., both are over 40).

So what's changed? Changes have been made to three sections in Title 29 of the Code of Federal Regulations:

- Section 1625.2 has been re-worded to clarify that employers may give preference to the older applicant in the scenario described above but states there is no requirement to do so and that this does not impact applicable state, municipal, or local laws that prohibit such preferences.

- Section 1625.4 has been similarly re-worded to clarify that employers may use language in job advertisements such as "over age 60", "retirees", or "supplement your pension"--this is exactly the opposite of what the rule stated previously.

- Section 1625.5 has been re-worded but no significant changes to content were made.

The revised regulations are available in either PDF or text format (PDF is much easier to read). Anyone with an interest in this area should read this Federal Register entry because it goes (briefly) into more detail about what the revision does and doesn't do.

EEOC press release is here.

An idea for checking false credentials

We all know how important it is to validate education and experience claimed by candidates. I've seen numbers as high as 50% for the frequency of, shall we say, embellishments, on resumes and applications.

Reference and backgrounds checks are the typical route for this check on applicant honesty, but they're time consuming and it can be challenging to get high quality information. Here's an idea to consider: how about using the interview as part of the background check process?

Have you considered asking questions like:

"I see you went to Texas A&M. Tell us a little about the types of courses you took and projects you worked on."

Job-related, specific, and forces the candidate to do a little more digging.

Or if you wanted to be more blatant:

"I see you went to UCLA. Tell us a little about the campus--where were the majority of your classes? What did you enjoy about the school?"

Not so job-related, perhaps, but certainly defensible as a check on their truthfulness.

Yes, deceivers could still prepare pat answers for these types of questions, but my guess is many won't and you'll save yourself a lot of time and headaches.

Thursday, July 05, 2007

A solution to discrimination lawsuits?

Dr. Anne Marie Knott, assistant professor at Washington University (Olin School of Business) has come up with an idea she says will reduce employment discrimination claims: an "anti-discrimination bond." That's bond as in financial instrument, not bond as in promise.

The "bond" is purchased by applicants and acts similarly to a 401k (payroll contributions are put into individual accounts) but with a hitch: the contributions are forfeited if the employee files a discrimination claim.

The idea seems to be that "litigious people" will find this distasteful and not apply to the organization. In fact, according to Knott, experiments using the bond may reduce litigation by 96%. And she's serious about this solution--she's even filed a patent for it.

I must admit this is pretty creative--the organization is not requiring people to sign away their right to sue (which has been upheld, but the EEOC can still go after you)--but instead tying it to a benefit. I'd be curious what the legal minds out there have to say about this one.

Looking at it from a different angle, I wonder what sort of message this sends to applicants? I know if I was asked to sign something like this I might wonder why they're even bothering--do they assume I'm a litigious type of person? Are they planning on discriminating against me?

Wednesday, July 04, 2007

Where does my traffic come from?

Given that it's a holiday (here in the U.S.), I thought I would post something on the lighter side...

Because I use Feedburner and Google Analytics, I'm able to see how my readers reach my home page. I thought it might be informative to show you how people get (t)here...

Here are, in descending order, the most popular ways people find HR Tests:

1. Through Wikipedia

2. By searching for Jobfox (a very promising job matching service that I wrote a post about)

3. From my colleague Michael Harris' ex-blog, EASI-HR blog (now he's over at HRMplus)

4. Through IPMAAC (a great professional organization devoted to public sector assessment)

5. Through recruiting.com (clearinghouse for recruitment-related matters)

6. From selectionmatters.com written by fellow blogger Jamie Madigan (who also writes for TIP)

7. Through searches for Hogan assessments

8. Through recruitingblogs.com

If you own a blog, you might consider becoming more visible through these avenues. Yahoo! Site Explorer will give you similar, albeit non-rolled-up, information.

Alternatively, you could now point out to me that I'm missing some obvious source of traffic :)

Other random information:

Visitor location: The majority of my visitors are from the states, with Pittsburgh being the most common source. I also get a fair amount of traffic from Bombay (Mumbai), India. Other visitor locations include Bangkok, Canada, Dubai, London, and Singapore (to name a few).

Search engine: Overwhelmingly Google. The other search engines don't even come close.

Web browser: Internet Explorer has the lion's share (83%) with Firefox at 12%.

That's all for now--thanks for reading!

Monday, July 02, 2007

JPSP, Vol. 92, Issue 6

There's a new issue of the Journal of Personality and Social Psychology out (volume 92, #6), with some juicy research for us...


First up, a fascinating study by Kawakami, et al. that may assist with efforts to eliminate or minimize discriminatory behavior. Participants in the study were trained to either pull a joystick toward themselves or push it away when shown pictures of Black, Asian, or White individuals. They then took the Implicit Association Test (a measure of how connected things in your memory are, used in this context to measure bias) or were observed for nonverbal behavior in an interracial context. Results suggested that simply engaging in approach behavior reduced "implicit racial prejudice" (as measured by the IAT) and increased "immediacy" in the nonverbal situation. Could this be incorporated into some type of training to reduce recruitment and selection bias? We'll see. (Mere exposure may be the more likely training route)


Second, an article that directly relates to the current focus in assessment circles on measures of training and experience (dovetailing with the increase in ATS). Moore & Small note that people generally believe they are better than others on easy tasks and worse on difficult tasks. The authors propose that these difference occur because people have much more information about themselves than about others. The result is even stronger when people have accurate information about themselves (!). The solution, it would seem, is to provide people with accurate information about how others perform.

What might this look like? A simplistic example would be instead of having people simply select categories such as Expert-Journey-Learning-Beginner, provide some data on how many folks tend to fall into each category. Unfortunately, I doubt this would be enough to overcome our built-in inaccuracy when it comes to self rating--but everything helps.


Last but not least, a study of a non-cognitive trait--and it's not one of the Big Five! No, this time it's grit, defined by Duckworth et al. as "perseverance and passion for long-term goals." Using various measures, the authors show that measures of grit added incremental variance to the prediction of a variety of criteria, including:

- Educational attainment among two samples of adults
- GPA among ivy league undergrads
- Retention in two classes of West Point cadets
- Ranking on the National Spelling Bee

Grit was not correlated with IQ, but was highly correlated with conscientiousness. It only accounted for about 4% of the variance in predicting the above outcomes, but the incremental validity added was beyond both IQ and conscientiousness. Is this practically meaningful? Depends on your point of view. If you're dealing with a large candidate group, or a particularly sensitive one (e.g., peace officers), could be worth a second look. Methinks more research is needed, particularly research on any subgroup differences.

Friday, June 29, 2007

New SIOP Journal in 2008

In 2008 SIOP will begin publishing its own scholarly journal, Industrial and Organizational Psychology: Perspectives on Science and Practice.

Here are the vitals:

Cost: Free to members (unknown fee for nonmembers)

Editor: Dr. Paul Sackett

Focus: Articles will cover basic science, applied science, practice, public policy, and (most likely) a blend.

Format: Focal article-peer response. Each article will be followed by 8-10 responses. From the website: "[Peer responses] could challenge or critique the original article, expand on issues not addressed in the focal article, or draw out implications not developed in the focal article. The goal is to include commentaries from various perspectives, including science, practice, and international perspectives. These commentaries will be followed by a response from the original author of the focal paper."

Publisher: Blackwell (which hopefully means we'll have online access)


The first two article abstracts are available here, with members having full access:

The meaning of employee engagement - Macey and Schneider (a particularly hot topic)

Why assessment centers don't work the way they're supposed to - Lance

Wednesday, June 27, 2007

EEOC issues FY 2006 Federal Workforce Report

The EEOC has released its Annual Report on the Federal Work Force for FY 2006 (10/o5-09/06).

For federal agencies it's a treasure trove of benchmark information covering everything from EEO policies to ADR statistics.

Not in the federal workforce, or otherwise find this a yawner? Check it out anyway. The report is filled with tips on topics such as:

- Reasonable accommodation procedures
- Sexual harassment policies
- Barrier analysis

- Improving participation rate of individuals with disabilities

And if you like tables and graphs...Woah, Nelly, you're in for a treat.

The report is available in HTML or PDF.

Tuesday, June 26, 2007

New blog to watch

Michael Harris, previously of EASI-HR Blog, has started a new blog titled HRMplus.

Dr. Harris is a professor at the University of Missouri-St. Louis where he teaches HRM. He has also served as an expert witness on discrimination issues, as a trainer, and a consultant. He presented at the most recent IPMAAC conference on Disparate Impact and Employment Testing: A Legal Update.

Check it out!

Monday, June 25, 2007

June 2007 IJSA

Yep, it's journal time again...the June 2007 issue of the International Journal of Selection and Assessment is out...

This time the articles fall into two main camps: those focused on applicant reactions to selection procedures, and those focused on personality testing.


Applicant reactions

First up, Landon and Arvey looked at ratings of test fairness by 57 individuals with graduate education in HRM. The test characteristics included validity coefficient, mean test score difference between minority and majority candidates, and test score adjustment for the minority group. Results? Fairness ratings depended almost as much on how raters rated as they did on the test characteristics! Implications? "Extensively cross validated with a global random sample of over 3 million with criterion-related validities exceeding .90!" may not be worth a hill of beans to some folk.

Next, Bertolino and Steiner with a similar study of test fairness using responses from 137 university students (ahh, where would our research be without students). Students rated 10 selection methods on 8 procedural justice dimensions. Results? Work sample tests were rated highest (as they usually are), followed by resumes, written ability tests, interviews, and personal references. All things that are expected by most job seekers. What wasn't perceived as well? Graphology--thankfully. The most important predictors of procedural justice were opportunity to perform and perceived face validity. Nothing earth shatteringly new, but some international confirmation of previous, mostly U.S., findings.

Speaking of international support for the importance of opportunity to perform and face validity among an international sample judging fairness of selection methods (say that five times fast), Nikolaou and Judge analyzed responses from 158 employees and 181 students in Greece. Methods rated highest (drumroll...): interviews, resumes, and work samples, across both groups. However, students reported more positive reactions to "psychometric tests" (i.e., ability, personality, and honesty tests) than did employees--an important distinction. Also, although there does appear to be individual differences in rating (see previous study), core self evaluations didn't appear to explain much.

In summary: Work samples and interviews--high fairness ratings and (potentially) high validity. A great combination.


Personality testing

First up in the personality section is Kuncel and Borneman's study of a new method for detecting "faking" during personality tests by looking at the particular way "fakers" respond. The authors found support for this method (sample was not described), with 20-37% of fakers identified. May not seem like a lot, but they had a false positive rate (incorrectly labeling someone a faker when they're not) of only 1% with a baserate of 56% honesty. Not too shabby. Interestingly, the "faker" pattern did not correlate with personality or cognitive ability test results.

Second, a very interesting study of the "dark side" of personality by Benson and Campbell. Using two independent samples of managers/leaders (N=1306 and 290), the authors found support for a non-linear (inverted U) relationship between "dark side" scores and assessment center ratings as well as performance ratings. So, for example, having a moderate amount of skepticism or caution is good--but too little or too much creates a problem. The instruments used were the Global Personality Inventory and the Hogan Development Survey.


Okay, there's a third category

Okay, one more study. de Mejier et al. analyzed results from over 5,000 applicants to Dutch police officer positions. The researchers were interested in rating differences when comparing ethnic minority and non-minority applicants. Results? Similar (if not more) assessment information was used to judge the two groups, but a large number of "irrelevant cues" were used to judge minority applicants. One other difference: when rating minority candidates, assessors relied more on the ratings of others. Another argument for a standardized rating process!

Saturday, June 23, 2007

JobAdWords Update

Update on the JobAdWords Survey

In a previous post I introduced a survey I'm using to answer some questions about the words used in job advertisements. The survey can be taken here, or if that's full, here or here.

Results so far? Based on responses from a mixture of HR professionals and job seekers (I'll break them out when I have a large enough sample):

The words most often seen in job advertisements:

- Motivated
- Flexible
- Customer-focused

The words least often seen in job advertisements:

- Independent
- Smart
- Friendly

The words with the most positive emotional response:

- Conscientious
- Reliable
- Strong work ethic

The words with the least positive emotional response:

- Flexible
- High-energy
- Creative

The words associated with highest probability of applying:

- Professional
- Detail-oriented
- Reliable
- Friendly
- Conscientious

The words associated with the lowest probability or applying:

- High-energy
- Flexible

So here's an interesting question (worthy of further research): do the words in job advertisements cause a reaction because they say something about what the job would be like, or because they cause people to self-assess? Or both? For example, "strong work ethic" received a high emotional response, but was not one of the highest-rated words when it came to applying. Hmmm...

Friday, June 22, 2007

June '07 JOOP

The June 2007 Journal of Occupational and Organizational Psychology is out, and while it has several articles of interest, there's really only one directly related to recruitment/assessment.

In the study, Piasentin and Chapman looked at how perceptions of person-organization (P-O) fit come about--whether they stem from feeling like the organization is similar to you, complements you, or some combination of both.

Using a sample of data from 209 employees "of various occupational and organizational backgrounds", the authors found support for both the similarity and complementary effects. In addition, perceptions of fit were found to correlate (and mediate the relationship) with several other important feelings, including:

- job satisfaction
- organizational commitment
- turnover intention

So what are the implications? How people perceive the match between their own skills, values, and goals and those of the organization matter--and not just to current employees but applicants as well. Organizations have to make sure they give applicants enough information to make this judgment, however. Too often job seekers are provided with minimal, or irrelevant, information about the position and the organization, such as long lists of tasks. Yes, people want to know what the salary is and where the job is located, but they also want to know who they'll be working with, what their career growth opportunities will be, and what the organization's take on work-life balance is.

This is low-hanging fruit from a staffing perspective, and organizations that get it are providing job seekers with this rich form of information.

...speaking of fit...there's a new book out on the topic, called Perspectives on Organizational Fit edited by Ostroff and Judge. It includes recruitment and selection as topics, but also covers others, such as leadership and teamwork.

Thursday, June 21, 2007

Police depts relax hiring standards

In response to serious recruiting challenges, many U.S. police departments are "lowering" their standards for hiring.

The reasons behind the shortage are many, including a strong job market, the Iraq war, and a high number of retirements.

Departments are using whatever means they have at their disposal, including upping their advertising. Case in point: while driving down 880 the other day in Oakland, CA, I noticed a sign promoting the $69,000 starting salary for Oakland Police Officers (and people wonder why it's hard to hire in the Bay Area).

The article cited above describes many steps departments are taking, some of which may initially seem like cause for concern. Let's take a look at them:

1. Forgiving minor criminal convictions, particularly old ones. If someone got busted 10 years ago for doing Ecstasy in college, and hasn't been in trouble since, is that still relevant?

2. Relaxing the 2-year college degree requirement, or allowing experience substitutions. I'm familiar with some research indicating a relationship between college education and officer performance, but if an officer has relevant experience (and performed well), this seems like a wash.

3. Raising the age limit. Age and job performance has been a hot topic in I/O psychology for a long time. While there are some declines over age (e.g., working memory), my reading is that they aren't practically significant in most situations. And we're talking about raising the limit to 40 or 44, not 85.

4. Relaxing fitness requirements. To me this comes back to plain 'ol validation. Granted, it's not always easy to determine where a pass point should be set (do they have to run 300 meters in 55 seconds or 56 seconds?), but do the study. Find out where a reasonable point would be. Run the numbers. See if it makes sense.

A lot of the concerns that go along with these changes--hiring people with low integrity, hiring people physically or mentally unable to perform the job--can be mitigated with good assessment, such as memory tests, physical ability testing, integrity testing, and reference and background checks.

Overall, I think this is a good thing--minimum qualifications (MQ) are often barriers to employment for certain ethnicities, women, and individuals with disabilities. And the situation is even worse when they aren't based on any rigorous study of the necessity for the MQs to being with.

On the other hand, I have heard anecdotally that similar changes in standards for U.S. Army recruits has resulted in more challenges for training.

What do you think--big deal or not?

Monday, June 18, 2007

2007 IPMAAC Conference Presentations

The 2007 IPMAAC Conference in St. Louis was great this year, with tons of informative and entertaining presentations, including great talks by Wayne Cascio, Bob Hogan, and Nancy Tippins.

The presentations are starting to roll in. Right now there's only one up--Michael Harris' great presentation titled Disparate Impact and Employment Testing: A Legal Update--but I expect more to show up this week and the next, so keep checking.

Next year I am co-chairing the program committee along with Carl Swander of Ergometrics. We're going to attempt to match the quality of this year's conference and plan on putting together an outstanding program. It's never too early to start thinking about presenting at next year's conference, to be held on June 8-11 in beautiful Oakland, California. Mark it on your calendars!

Saturday, June 09, 2007

Project JobAdWords

I'll be in St. Louis at the IPMAAC conference for most of the week, then taking a little time off...In the meantime I thought I would gather a little data...

I know there's research out there that looks at applicant reactions to various aspects of a job advertisement, such as information on selection procedures, pictures, and clear descriptions of work climate. But I'm not aware of any that specifically looks at the effect of WORDS commonly contained in job ads (although there is a recent study about word frequency).

You know the ones I'm talking about:
- "motivated"
- "creative"
- "works well under pressure"
etc.

But what exactly do these words mean to a reader? What reaction do they cause?

Today I'm starting Project JobAdWords, an attempt to try to answer these questions.

I've created a very brief survey designed to shed a little light on what these words mean. I would greatly appreciate your participation in this project. Simply go to one of the following websites--if survey #1 is full, go to survey #2, if survey #2 is full, please try survey #3:

Survey 1
Survey 2
Survey 3

As soon as I have enough respondents--judged completely arbitrarily and in no way using statistical sophistication--I will post the results. If I get enough results, I may continue this project by looking at other similar issues.

Thank you! Please pass the word.

Thursday, June 07, 2007

March '07 issue of Journal of Business and Psychology

The March issue of the Journal of Business and Psychology has two articles we should take a look at...

1) A study by Topor, Colarelli, and Han of 277 HR professionals found that evaluations of hypothetical job applicants were influenced by the traits used to describe the applicant as well as the assessment method used to measure those traits. Specifically, candidates described as conscientious were rated highest, as were those assessed using an interview--and the combination of the two was the highest-rated combination. (Other traits included intelligence and agreeableness, other assessment methods were paper-and-pencil test and assessment center)

How does this stack up against known evidence? Could be better. In general (and there are variations depending on the job), intelligence has been shown to be the best predictor of job performance. The value of a particular assessment method, on the other hand, depends not only on the job, but on the nature of the instrument as well as the construct being measured. But the research on measuring personality using interviews is relatively new, and certainly not as established as evidence supporting paper-and-pencil measures of cognitive ability (or personality, for that matter). Another example of HR folk not knowing the evidence as well as they should.

2) A meta-analysis by Williams, McDaniel, and Ford of compensation satisfaction research. The authors looked at 213 samples from 182 studies and came up with some interesting results.

- First, measures of satisfaction with direct pay (e.g., pay level, pay raises) were highly related to one another, while satisfaction with benefits showed only modest correlations with direct pay (suggesting people see direct pay and benefits as distinct compensation categories).

- Second, both the perception of the pay raise as well as the objective amount of the pay raise were important in predicting pay raise satisfaction. This fits well with what we know about the importance of procedural justice.

- Third, a modest, negative relationship was found between employee costs and benefit satisfaction. This suggests benefit costs may not be a particularly important factor when trying to lure candidates. It also suggests that asking employees to pick up more of their benefit costs may be a viable strategy as long as attention is paid to other forms of compensation (which apparently have a larger impact on overall satisfaction).

Wednesday, June 06, 2007

Recruiting metrics

One of my readers asked me to address some recruiting metrics, which I think is a good topic to visit.

Although there are some strong feelings out there about the usefulness of metrics, the fact is that traditional measures of recruiting activity are still out there being used, both as measures of success and for planning purposes.

The two benchmarks that were specifically asked about were:

- Number of fills per recruiter
- Number of hours per recruitment

Unfortunately a lot of the benchmark recruiting metrics out there are not free. Several organizations offer detailed benchmarking data, but ya gotta pay for it. Examples include:

- Staffing.org's Recruiting Metrics and Performance Benchmark Report ($425)

- Society for Human Resource Management (SHRM)'s Human Capital Benchmarking Service ($425 for non-members)

- The Saratoga Institute publishes a Human Capital Effectiveness Report

Even so, I've gathered what data I could get my hands on, so let's take them one at a time.


Fills per recruiter

How many fills, or placements, does a typical recruiter make? Of course it depends on the time frame and what type of placement we're talking about, which makes this statistic challenging to nail down. But here's what I found:

- IPMA-HR's 2006 Recruiting and Selection Benchmarking Report: The median number of recruitments per HR staffer was in the 4-6 category, closely bordering on the 7-10 category, so six is probably a good overall estimate given this data.

The other sources I found were more anecdotal. For example:

- JBN Consulting notes a typical recruiter makes .75 placements per month

- Cluff & Associates state: "A dedicated recruiter in a corporate environment can average 8 fills per month if activity levels warrant. Agency recruiters, however, are likely to generate 2 or 3 placements per month."

-
A Google search on the topic brings up quite a few recruiter job advertisements, which gives us some idea of the expected workload. In general, 1-3 fills per month seems a standard expectation.

Conclusion? The typical number of fills per recruiter seems to vary quite a bit depending upon the type of job, job market, etc. For example, Dave Lefkow noted a few years back that 100 fills a month for customer service jobs was considered successful, while 5-10 analyst/developer fills a month might be acceptable. If I had to pick an "average" number, I'd go with the IPMA-HR figure of six (although consider that's public sector organizations).


Hours per recruitment

Fortunately there's a lot more information about benchmarks for how long it takes to conduct a recruitment (here used synonymously with "time to fill"), although here too the figure varies with the type of recruitment:

- Surveys conducted by the Employment Management Association (affiliated with SHRM) consistently show an average time to fill of 42-45 days. Source

- 2004 Workforce Diagnostic System Benchmark by the Saratoga Institute indicated 48 days.

- 2006 SHRM metric for same-industry hires of 37 days (from opening of requisition to offer acceptance) Source SHRM also offers customized reports (for a fee), available here. Samples indicate median time to fill of 35 and 40 days, depending on industry.

- 2006 Corporate Executive Board survey indicated 51 days Source: The Economist

- 2006 IPMA-HR benchmark report: average of 49 days between vacancy announcement and first day on the job, with a low of 44 for labor/maintenance and a high of 57 for public safety

- Staffing.org and HR Metrics Consortium's 2003 Recruiting Metrics and Performance Benchmark: 70 days (this includes time between offer acceptance and showing up to work). Source

Conclusion? Again, depends on the job (and how you define the metric), but we can estimate around 40 days between vacancy announcement and offer acceptance, with another 10-20 days between offer acceptance and first day on the job.

If you have additional data, please share! Inquiring minds want to know...

Tuesday, June 05, 2007

Feds host virtual career fair

The U.S. Office of Personnel Management (OPM), in conjunction with the Partnership for Public Service, will host a "virtual career conference" to highlight opportunities with the federal government today, Wednesday, and Thursday. You can view the live web cast here.

The event will kick off at 11 a.m. on June 5th with an overview of federal employment by the Director of OPM, Linda Springer. Subsequent panel sessions will focus on finding and applying for jobs, insight from new federal employees, student programs, IT jobs, opportunities in medicine and public health, how to host a career fair, and several other topics.

According to the news release, videos of all twelve panels will be available through both OPM and the Partnership for Public Service for the remainder of the year. The event is specifically targeted at the 600 colleges and universities around the U.S. who participate in the Call to Serve initiative.

Kudos to these organizations for being creative about highlighting opportunities in the public sector and taking advantage of technology to get the word out.

Monday, June 04, 2007

Summer 2007 Personnel Psychology + free content

The Summer 2007 issue of Personnel Psychology (v. 60, #2) is here and it's got some good stuff, so let's jump right in!

First off is the aptly titled, A review of recent developments in integrity test research by Berry, Sackett, and Wiemann, the fifth in a series of articles on the topic. This is an extensive review of research on integrity tests since the last review, which was done in 1996. There's a lot here, so I'll just hit some of the many highlights:

- It appears that integrity tests can vary in their cognitive load depending on which facets are emphasized in the overall score.

- It is likely that aspects of the situation impact test scores (in addition to individual differences); more research is needed in this area.

- Although there have been no significant legal developments in this area since the last review, concerns have been raised over integrity tests being used to identify mental disorders. The authors do not seemed concerned, as these tests (e.g., Reid Report, Employee Reliability Index) were not designed for that purpose thus likely do not violate EEOC Guidelines.

- Research on subgroup scores (e.g., Ones & Viswesvaran, 1998) indicate no substantial differences on overt integrity tests; no research has addressed personality-based tests.

- Test-takers do not seem to have particularly positive reactions to integrity tests, although this appears to depend upon the type of test, items on the test, and response format.

Next, Raymond, Neustel, and Anderson investigate certification exams and whether re-taking the same exam or a parallel form results in different score increases. Using a sample of examinees taking ARRT certification exams in computed tomography (N=79) and radiography (N=765), the authors found no significant difference in score gains between the two types of tests, suggesting exam administrators may wish to re-think the importance of alternate forms for certification, particularly given the cost of development (estimated by the authors at between $50K and $150K). The authors do point out that the generalizability of these results is likely limited by test type and examinee characteristics.

Third, Henderson, Berry, and Matic investigate the usefulness of strength and endurance measures for predicting firefighter performance on physically demanding suppression and rescue tasks. Using a sample of 287 male and 19 female fire recruits hired by the city of Milwaukee, the authors found that both measures (particularly strength measures such as lat pull-down and bench press) predicted a variety of criteria, including a roof ladder placement exercise, axe chopping, and "combat" test. The authors suggest continued gathering of data to support the use of these types of tests (while acknowledging the ever-present gender differences), and discuss several problems with simulated suppression and rescue tasks, now used by many municipalities in light of previous legal challenges to pure strength and endurance measures.

Lastly, LeBreton, et al. discuss an alternate way of demonstrating the value of variables in I/O research. Traditionally researchers have focused on incremental validity, essentially the amount of "usefulness" that a variable adds to other variables already in the equation. (Allows you to do things like determine if a personality test would help you predict job performance above and beyond the test(s) you already use.) Instead, the authors present the idea of relative importance, which shifts the focus to the importance of each variable in the equation. Fascinating stuff (and far more than I can describe here), and something I'd like to see more of. I believe the authors are correct in stating it would be much easier to talk to managers about how useful each test in a battery is rather than the fact that overall they predict 35% of performance. The article includes a fascinating re-analysis of Mount, Witt, & Barricks' 2000 study of the use of biodata with clerical staff.

----

This issue also includes reviews of several books, including the third edition of Ployhart, Schneider and Schmitt's Staffing Organizations (conclusion: good but not great), Weekley and Ployhart's Situational Judgment Tests (conclusion: good as long as you already know what you're doing), and Griffith and Peterson's A Closer Examination of Applicant Faking Behavior (conclusion: good for researchers, not so good for managers).

---

But wait, there's more...the Spring 2007 issue, which had some interesting stuff as well, is free right now! So get those articles while you can. Hey, it's worth surfing over there just for McDaniel et al.'s meta-analysis of situational judgment tests!

Saturday, June 02, 2007

Google buys FeedBurner; introduces Street View

A couple news items today about Google.

First, they continued their buying spree and purchased FeedBurner, a popular feed syndicating service (and the one I use). This will allow Google to advertise in new ways, and also gives them somewhat of a blog end-to-end now that it owns both Blogger and FeedBurner. More information about the purchase is available on FeedBurner's blog, and they have this FAQ. Presumably this means a marriage of their analytic tools, which hopefully is good news for anyone that publishes content.

Implication for us? If you already publish a blog (or other type of feed) or are thinking about it, you'll want to keep close tabs on what this purchase will mean for publishers (e.g., ad distribution, fees, etc.).

Second, Google has introduced Street View, an add-on to Google Maps that allows for 360-degree viewing at the street level. You can even follow arrows that will lead you down the street. Right now it's available only in San Francisco, Las Vegas, Denver, New York, and Miami.

Why do we care? Seems to me a good way to preview the local area for potential applicants. I'm all about the realistic job previews. Also, probably a good opportunity for us to see how our area is being presented--what do candidates see when plopped down in front of our buildings? Would you want to be there?

Friday, June 01, 2007

Becoming passive employers

Let's take a moment and think about what job search could be.

Right now, job search is static. Someone searches for a job, and either a vacancy exists or it doesn't. But what if we were a little more creative?

What if instead of getting "zero results for your search", the candidate received something like:

There are no current openings that match your search. However, the following positions exist that may have openings in the future.

What followed would be a detailed description of current positions in the organization that matched the search criteria--jobs people actually had. And you would allow people to submit a job interest request so they would be notified when that job (or similar job) became open. Yes, some systems already have job interest requests, but too often it's based on broad job titles and it fails to provide the rich information a job seeker needs (e.g., who they will work with, learning opportunities).

What else could we do with this feature? We could profile the individuals that are in the current job. Okay, maybe not everyone, but a sample. At the very least we could provide a basic job description (and not a boring one).

This idea fits with a concept I think we all need to focus more on. In addition to seeking passive candidates, we should be passive employers. Passive job seekers aren't looking for a job, but they could be. Passive employers don't have that particular opening--but they could. But unless you tell candidates that, how will they know? How do they know that a perfect match exists in your organization, and if they just had waited another week to search, they would have seen it?

Why do we make applicants the servants of the ATS, not the other way around?

Let's take this a step further.
Let's say I'm an attorney in Seattle looking to relocate to Boston. I know I'd like to work for a smallish firm with decent billable hours, co-workers that know their stuff and are good at their jobs but value work-life balance.

How the HECK am I supposed to find that firm? Sure, I can look for current vacancies on job boards. Or maybe I just happen to know someone who works for such a firm and they have an opening. Or I might be able to find some information through a Google search or services such as Vault or Hoover's (although that information is very limited, you still have to know the company name, and information on public sector agencies is anemic). But that'll only get me so far. Then what?

There is no general database of employer qualities to search through (sites like Jobfox are trying a similar idea but it's still based on vacancies). No easy way to punch in the above criteria and have a system spit out, "Here are all the firms that meet your criteria. Here are the ones that currently have openings, here are the ones that don't currently may may in the future."

People search is getting more and more sophisticated. What about employer search? If we expect applicants to take an active role in managing their career, we should give them the information they need to do it. We can, and should, do better.