Big research update

It's been a while since I provided a research update, so let's take a look at some recent highlights:

The March 2017 issue of the International Journal of Selection and Assessment (IJSA) (free right now!):

•  Some support for individual psychological assessment in a sample of middle- and senior-level civil service managers in the UK
• Concerns about personality measures that are non-contextualized (i.e., not tailored to the job)
• A study of the personality traits of high potentials (conscientiousness came out #1)
• Support for the equivalence of GMA tests on multiple devices, including mobile

The June IJSA:

• A fascinating twin study that found support for a strong relationship between counterproductive work behaviors in personal life and work.
• Applicants higher in conscientiousness and emotional stability may be more likely to re-test and improve their GMA scores
• A study of undergrads suggesting that Big 5 score results from internet-based tests and in-person tests are equivalent
• You may be able to decrease the number of candidates that withdraw from selection by engaging them more fully and ensuring your processes are fair
• Other-reports of personality appear to be fakable, but less so than self-reports

Vol 2(1) of Personnel Assessment and Decisions:

• Trusting, intelligent raters may be more able to pick up on impression management during interviews
• A study using the limited data available suggests applicants prefer in-person over technology-mediated interviews; interview ratings were also lower. Beware varying interview mode across applicants!
• A study of unproctored internet cognitive ability tests suggests cheating is common, and the researchers strongly suggest implementing technological precautions
• Hiring managers may resist standardized testing in part because they feel it reduces their usefulness/impact
• Free personality test items that can be used and mapped to the PPRF, a personality-based job analysis tool

April Psychological Bulletin:

• A meta-analysis showing that reaction time measures can be useful in detecting deception, but are themselves susceptible to counter-measures

March Journal of Applied Psychology:

•  A review of 100 years of selection and recruitment research in this journal

May Journal of Applied Psychology:

• A study of a large group of applicants suggests that recruiting candidates earlier (e.g., high school), and with more diagnostic methods, can influence their investment in their career and make them better candidates
• Using several computer simulations, researchers found that incumbent-based estimates of range restriction may be downwardly biasing criterion-related validity values, and can produce evidence of differential validity were none exists

March Journal of Organizational Behavior:

• The Big 5 factors of neuroticism, conscientiousness, and agreeableness may prevent supervisors from turning frustration into abusive supervision

May Journal of Organizational Behavior:

• Big age differences between subordinates and supervisors can negatively impact firm performance—if they're discussed

June Journal of Business and Psychology:

• Is the KSAO of communicating via computer just another version of face-to-face communication? According to this study, they are distinct.
• Do millennials have a worse work ethic than older employees? It appears not based on this meta-analysis

That's it for now!

Research update

It's that time, February research update--and there are some really fascinating studies out there:

First the March 2013 issue of IJSA:

- Procedural fairness is an important aspect of organizational attraction, especially for rejected applicants.

- Does personality type impact perceptions of the selection process?  According to this study and their measures (Bohemian is a personality type?), sometimes.

- Do you use SEM to generate a SED and ultimately score bands? Then you will definitely want to read this piece, which suggests the traditional method is flawed.

- Emotional intelligence continues to be a hot topic.  In this study the authors describe the development of a SJT-based measure.

- Staying on the topic of personality, this study adds to the debate over the utility of constructs vs. facets in predicting performance--this time with conscientiousness predicting police officer performance.

- P-O fit is often used to explain why people apply for certain jobs.  In this study researchers found that values that applicants find attractive are particularly important (as opposed to neutral or aversive aspects).

- I've written about automated reference check systems before.  In fact I've written about the company that the first author of this study is from (SkillSurvey).  What is partially a demonstration of the value of their product also adds to our knowledge of reference checks by finding no demographic differences and the ability to predict involuntary (but not voluntary) turnover.

- How big of a deal is it when it is discovered that an applicant "embellished" on their resume?  Depends on how much you already liked them.

- In this study of applicants in Iran, the authors found that the usefulness of a web-based application system was more important than how easy it was to use.  So basically looks are important but not sufficient.


Next, the January issue of the Journal of Applied Psychology: 

- Construct validity has always been a sticky wicket.  The authors of this study take an interesting approach, using video-based testing to illustrate a method for supporting construct validity.

- This study explores the variance in assessment center ratings stemming from various sources, and the authors argue that existing literature has masked this variance.

-The authors of this study show how the application of a particular bootstrap method applied to meta-analyses increases accuracy.

- Speaking of meta-analysis, the authors of this study demonstrate that the differences between coefficient alpha and composite reliability in actual data sets is quite small.


On to the January issue of the Journal of Applied Social Psychology, which as of the time of this post was available for free!

- Similar to findings in other areas, the impact of negative publicity is stronger than positive publicity on organizational attraction, according to the authors of this study.  The authors also found in these cases, organizations would do well to provide detailed recruiting ads to mitigate the impact.

- Age discrimination seems to be increasing in popularity as a research topic (which makes sense from a demographic perspective).  In this study, the authors found raters discriminated against older and younger applicants, with the former being the least likely to be hired.

- Here's an interesting one: in this study, people of color perceived White women to be, among other things, ditsy, shallow, privileged, and appearance focused.  Stereotypes know no color boundaries.


Let's take a look at the February 2013 issue of the Journal of Personality and Social Psychology:

- First, a fascinating look at how generalizable the five-factor model of personality is across cultures.  Specifically, the authors looked at a largely illiterate, indigenous culture in Bolivia.  They failed to find support for the widely used model, and suggest that the structure of personality may differ across societies.

- Second, a study of sex and gender differences that points out the importance of considering the construct of gender (as opposed to the binary concept of sex) for a variety of behavioral and psychological indicators, including big five personality.


Let's move now to the February 2013 issue of Psychological Science, where:

- In this study, the authors found that age differences in cognitive functioning within cohorts were often as large as those between cohorts, which calls into question the cohort explanation for age differences.

- In a study that I think has implications for personnel assessment, the authors found that raters of M.B.A. applications that had already given several high recommendations were less likely to do so later in the day.  Another bias to watch out for!


Last but not least, there was an article recently in the New York Times about how important referrals are for some organizations and the increased importance this places on social networking sites.  There was a reference to a study done in August of 2012 on source of hire that I hadn't seen before, which found that not only were referrals more likely to be hired, they had longer tenure--which is consistent with other research I've seen.  Unfortunately, no word about performance differences.

That's all for now!

Research update

 
Cold/wet yet?  Well sit back, have some hot tea, and let's catch up on our research...

Let's start with the biggie: the December issue of IJSA.

-  Juggling selection quality and adverse impact continues to be tricky.  The authors in this article suggest an optimum combination.

- Reflecting results from the turnover literature, this study found a relationship between perceptions of promotion practices and organizational justice and job satisfaction.

- Speaking of justice, the authors of this study found that the relationship between perceptions of distributive justice and intentions to recommend an employer were moderated by applicant affect.

- Job seekers would to well to keep up their psychological well being and self-esteem (easier said than done, right?).

- Back to fairness.  It's reasonable to think that ethnic minority applicants may not perceive video resumes well (due to the increased salience of their minority status).  But at least in this study, that assumption was not supported--although it depended on ethnic identity and language proficiency.

-  Honestly, I'm not one for the pure concept of "multi-tasking": in my experience people perform in serial, not parallel.  But that doesn't stop people from researching the predictive validity of multi-tasking assessments.

- Faking of personality inventories has been one of the hottest topics in assessment for years, but is often framed as dichotomies.  This study looks to bring some needed complexity to the issue using qualitative data.

- We all know it can be challenging to get hiring manager to give up their unstructured interviews of questionable validity.  Interestingly, according to this study, the more a hiring manager has used unstructured interviews, the less open they are to change.  I don't know if this is a causality issue, a mediating variable issue, or more evidence of the inability to accurately judge one's abilities. 

- Need more evidence for discrimination that occurs during resume screening?  Here ya go.

- Understanding why certain individuals perform better during interviews is a needed area for study.  In this article, the authors demonstrate the importance of simply being ability to generate ideas, rather than analyzing the situational requirements.

- This study investigates another area needing more attention: the selection into medical training and education programs.  The authors found strong predictive support for SJTs but they came with lower face validity.

- Last in this issue is another interesting study, this time of 360-ratings of innovation.  Interestingly and unlike a lot of other research on self-perception, self-ratings were lower than overall observer ratings.  However, the situation got more complex when the authors separated and analyzed by level of self-rating.


Next the November issue of Journal of Applied Social Psychology:

- One of the most pervasive (yet bizarrely under-discussed) areas of discrimination in the workplace is age discrimination.  This study illustrates some of the stereotypes held of younger and older workers--by both groups.  (Spoiler alert: you'll find out how accurate some of these are in just a second)

- I've been waiting for this one, partly because I love hearing about how bizarre and non-face valid some puzzle-based interviews are.  This particular study was looking at perceptions of these interviews compared to a behavioral interview.  Results?  The puzzle-based interviews were consistently less popular.  Oh, did I mention that they didn't work as well?


How about the Winter issue of Personnel Psych?

-  Why do certain applicants withdraw from the recruitment process?  This study suggests a relationship with organizational identification.

- Okay, back to stereotypes about older workers.  Whereas the earlier study looked at what the stereotypes are, this one looks at whether they're true.  The answer: no, but for one: older workers are less willing to participate in training and career development.

- Turns out it's not just the unemployed that are frustrated by the job search process--currently employed individuals feel the same in many ways.  Boy it's too bad we don't have a giant shared database that is able to match job demands with worker abilities...wait...


The November issue of Journal of Applied Psychology has a couple gems:

- Evidence for the predictive validity of the external manifestations of personality as well as the associated implicit motives.

-  A reminder that what makes for effective leadership behavior depends on the culture.


The latest issue of Personnel Review has an interesting research article on utility analysis, where the authors reiterate how challenging it can be to communicate UA information (hint: carrot and stick approach may work best).


There are a couple good ones in the November issue of Psychological Science.

- Multiple-choice tests have been beat up in the past for being nothing more than tests of recognition (rather than productive retrieval).  This study presents evidence that refutes that assumption.  Go multiple-choice!

- Conspiracy theorists take note: governments may be less likely to use the assumed relationship between genetic testing and intelligence to pigeon-hole us into tracks.  Why?  Because, at least according to this study, there appears to be little evidence connecting the two.


Okay, this one is pretty cool--in a slightly scary way.  The authors were looking at the impact that virtual avatar attractiveness has on interview ratings.  Turns out our bias toward attractive people is so strong it extends to the virtual world!  Of course maybe I should have seen that coming...I mean, ever read a comic book?  (hat tip)


Still with me? Last but not least, some disturbing new evidence regarding significance testing and potential publication bias (hat tip).  I'm guessing most of you won't be surprised at the finding.


I don't know if I'll have another update before the end of the year, so if I don't, happy holidays to everyone!

Research update: From item context to signaling theory and more

Here are some research articles from the last couple months:

Grand et al.'s study showed that adding a job-relevant context to test items--even under explicit stereotype threat--had either beneficial or no effects on test performance and test perceptions among female test takers. More evidence of the benefit of tailoring items to the particular position being tested for.

Impression management during assessments is often considered to be a negative thing--i.e., a source of error. But as Kleinmann and Klehe point out in their study of interviewee behavior, it may be an additional source of validity, and can be related to performance. At the very least it indicates that the person knows enough to alter their behavior to fit the job!

Celani and Singh provide a literature review of the role of signaling theory in applicant attraction (making inferences about important aspects of the job/organization from characteristics of the recruitment) and how social identity interacts to impacts attraction outcomes.

Last but not least, Soto et al. with a fascinating study of age and personality characteristics. Over a million individuals participated over the web, and the authors highlight several key results, such as late childhood/early adolescence being key periods for age trends, strong maturity and adjustment trends over adulthood, and the importance of looking at facet-level results.

Generational differences in work values: Fact or fiction?

There's been a lot written over the years about so-called "generational differences" among different age groups--e.g., Baby Boomers, Gen X, and GenMe (i.e., Gen Y, Millennials, Net Generation). Various authors have claimed important differences, such as GenMe valuing altruistic employers and social experiences much more than, say, Boomers. Take a look at the business section of your local bookstore and you're sure to find examples.

The problem isn't with the issue--if there truly are differences, they would have important implications for attracting and retaining different segments of the workforce. The problem is with the data.

Turns out most of the previous research is either qualitative (anecdotes) or based on cross-sectional studies done at a single point of time. The problem with these studies is they make it impossible to separate generational differences from career stage differences. In other words, younger applicants/employees may indeed differ from older ones at any point in time--but that could be purely due to factors that impact one's age, not factors related to being born or experiencing a particular point in time.

Luckily for us, in the September issue of the Journal of Management, Twenge, et al. report the results of a longitudinal study that allows us to answer these generational questions with some authority.

The authors used data collected from a nationally representative sample of over 16,000 U.S. high school seniors taken in 1976, 1991, and 2006 (from the Monitoring The Future project).

The results may surprise you. Let's look at each of the work values studied by the authors:

Leisure (e.g., vacation, work-life balance): This became progressively more important over the generations, with GenMe valuing it the most. The difference between GenMe and Boomers was the largest reported in the study (d>.50).

Intrinsic (e.g., interesting and challenging work): While GenY did not differ significantly from Boomers, GenMe were significantly less likely to value this compared to either GenY or Boomers.

Altruistic (e.g., ability to help others and society): While it's commonly reported that GenMe values this more highly than previous generations, results did not support this. No significant differences were found between the three groups.

Social (e.g., job gives feeling of belonging and being connected): This is another area where some have suggested that with the skyrocketing success of sites like Facebook, the younger generations more highly value--indeed, insist on--a workplace that allows social interaction. The results? Not so much. In fact, GenMe placed less value on this compared to both GenY and Boomers.

Extrinsic (e.g., job pays highly or is prestigious). This is an interesting example of non-linearity. Turns out this value peaked with GenY. While GenMe valued this more than Boomers, the difference was more pronounced between Boomers and GenY.

Among the items with the biggest differences were:

- GenMe valuing having 2+ weeks of vacation compared to Boomers
- Boomers valuing a job that allows you to make friends compared to GenMe
- GenY valuing having a job with prestige/status compared to Boomers
- GenMe reporting that work is just a way to make a living compared to Boomers
- GenX valuing being able to participate in decision making compared to Boomers

Overall, intrinsic reward items had the highest means across the generations, with a job that is "interesting" having the highest item mean. Altruistic and social values were also valued more highly, with extrinsic rewards having lower mean values and leisure rewards having the lowest.

The authors summarize the results by saying the data suggest "small to moderate generational differences." If you aren't surprised, kudos to your observational skills. At the very least this is important data to consider when evaluating your recruiting and retention efforts. And it certainly calls into question some of the conclusions being drawn in the popular press.

By the way, a full version of the article is available (at least it was at the time I published this) here.

July 2010 J.A.P.

A new round of journals is out, so let's start with the June issue of the Journal of Applied Psychology.

First up, Schleicher et al. looked at whether there were demographic differences in how much candidate scores improved upon retesting. Turns out there were several. Whites showed larger improvements than Blacks or Hispanics on several assessments, particularly on written tests. Women and applicants under 40 showed greater improvements than men and applicants 40+. Implications? In some situations allowing applicants to retest may exacerbate adverse impact.

Next, an important piece by Aguinis et al. (that you can read here) about test bias. This follows on the heels of the June IOP articles on the same topic and seems to represent a resurgence of interest in a topic that seemed dormant. In this article the authors report the results of a very large Monte Carlo simulation (billions and billions of data points) where they found that if bias is measured using slope-based techniques, it's likely to go undetected, and intercept-based bias favoring minority group members is likely to be found when in fact it does not exist. This study, combined with points made in the IOP article suggest that some of the "established" conclusions regarding test bias may not be as solid as we thought.

Third, for those of you interested in differential functioning (of items or scales), you should check out the piece by Adam Meade where he presents a taxonomy of potential differential functioning effect sizes and also describes a software program created for computing the indices and graphing differential functioning.

Next, a piece by Wang et al. on locus of control. Importantly, they found that when locus of control (LOC) is specific to work-related issues, there are stronger correlations between LOC and work-related criteria such as job satisfaction and commitment. Similarly, when LOC is defined more broadly to include non-work issues, there are some stronger correlations with non-work criteria such as life satisfaction. Implications? Much like research on personality items, specifying a work-related context would seem to increase the predictive power of LOC measures.

Last but not least, an important article on counterproductive work behavior (CWB) and organizational citizenship behavior (OCB) by Spector, et al. CWB and OCB seem like they should be opposites of each other--one demonstrated by disengaged, unhappy workers, the other by engaged, happy ones--right? Not so fast. The authors report the results of an experiment that suggest that the concepts are unrelated and do not necessarily have opposite relationships with other variables. The authors also recommend that when measuring these behaviors, frequency of performance be used rather than level of agreement.

Research update

Here are a few research articles you may have missed. Just links this time because...well, I'm really enjoying World Cup.

McGrath, et al.: Evidence for response bias as a source of error variance in applied assessment.
(May Psych. Bulletin)

Hulleman, et al.: A meta-analytic review of achievement goal measures: Different labels for the same constructs or different constructs with similar labels? (May Psych Bulletin)

Thornton & Potemra: Utility of assessment center for promotion of police sergeants. (March Public Personnel Management)

Bright: Why age matters in the work preferences of public employees: A comparison of three age-related explanations (March Public Personnel Management)

Thomas, et al.: Employee proactivity in organizations: A comparative meta-analysis of emergent proactive constructs (June JOOP)

Bjørkelo, et al.: Predicting proactive behaviour at work: Exploring the role of personality as an antecedent of whistleblowing behaviour (June JOOP)

Zibarras & Woods: A survey of UK selection practices across different organization sizes and industry sectors (June JOOP)

Is recruiting using SNS discriminatory?

I keep reading/hearing about how recruiting using social networking sites (SNS) opens employers up to discrimination lawsuits because of who uses the sites. For the most part, this just plain isn't true.

A recent Pew study is the latest to show that when it comes to using SNS like Facebook, MySpace, and LinkedIn, you really should have one primary demographic concern when it comes to ensuring a diverse candidate pool: age.

Not gender, at least not in traditional sense. While four years ago SNS users tilted slightly male (55%), the balance has essentially flipped today (54% female).

Not race, there simply do not appear to be generalizable differences in racial groups when it comes to these sites (in fact I've seen some data that suggest the user base on these sites is more diverse)--but things change, and this may vary with particular sites, so keep an eye on this one.

But when it comes to age, SNS users are disproportionately younger than the overall Internet population. In the words of the Pew report, "[this] doesn't mean that more older adults aren't flocking to SNS--they are--but younger adults are ALSO flocking to the sites, so the overall representation of the age cohorts in the SNS user population has actually gotten younger."

One demographic difference I don't see a whole lot about: disability status. Are individuals with disabilities more/less likely to use SNS? I think that's an important question we need to address if we're truly trying to diversity our candidate pools.

Research round up

There has been so much good research coming out that rather than attempting to write a summary for each of them, allow me to simply present titles and links for you. I'll bet you see something you'll want to follow up on:

Cognitive and emotional processes in individuals and commercial web sites (how people respond to banner ads)

Occupational embeddedness and job performance (interesting results for people that are strongly linked to their career field)

Age stereotypes in the workplace: Common stereotypes, moderators, and future research directions (qualitative summary)

The hidden prejudice in selection: A research investigation on skin color bias

Does stereotype threat affect test performance of minorities and women? A meta-analysis of experimental evidence (really good stuff about moderators)

Stereotype threat reinterpreted as a regulatory mismatch.

Does socioeconomic status explain the relationship between admissions tests and post-secondary academic performance? (short answer: no)

Similarity and assumed similarity in personality reports of well-acquainted persons. (depends on which HEXACO factor you look at)

Not all conscientiousness scales change alike: A multimethod, multisample study of age differences in the facets of conscientiousness. (hint: there's something special about orderliness)

Enjoy!

Supreme Court clarifies ADEA burden shifting

On June 19th the U.S. Supreme Court made several employment-related decisions. Of most interest for us is their decision in Meacham v. Knolls.

The case involved workers over 40 who were suing over their layoffs. They claimed they lost their jobs due to age discrimination, claiming a violation of the Age Discrimination in Employment Act (ADEA). Although they claimed both disparate treatment and disparate impact, the important issue here is the latter--employment decisions that may not intentionally discriminate but have that effect.

How the court ruled is closely tied to its 2005 decision in Smith v. City of Jackson, in which they held that adverse impact cases could be brought under the ADEA, but employers could prevail if they could show (per the language of the ADEA) that the employment decision was based on a "reasonable factor other than age" (RFOA).

So what was the decision? The court made it clear that the employer in these cases bears both the burden of production and the burden of persuasion that the employment decisions were based on a RFOA. This is similar to other adverse impact discrimination cases, such as those brought under Title VII, where an employer must show their practice was "job related and consistent with business necessity."

So what does this mean? It doesn't mean a new requirement. It reinforces that all employment decisions--hiring, firing, and everything in between--should be based on logical, non-discriminatory reasons. The fact that the employer may face a slightly easier hurdle in ADEA disparate impact cases compared to, say, race or gender cases, is practically insignificant.

Important note: the plaintiffs in this case provided expert testimony that employee scores on "flexibility" and "criticality" had both the most manager discretion and were tied the strongest to outcomes. Words like these are often invoked in age discrimination cases (a jury can easily see how these types of words might be proxy for "young"), and employers are wise to strongly consider in hiring and firing situations whether the rating factors are tied to benchmarks and can be shown to be important for success on the job.

Have teens lost the ability to write?

Have teenagers lost the ability to form a coherent sentence? It's something I've been hearing off and on the last few years from hiring managers and teachers. Generally I chalk it up to the normal generational differences, but given results from a recent survey I may have to change my tune.

The survey results come from a new study out by the Pew Internet and American Life Project--"Writing, Technology, and Teens." The researchers conducted focus groups and a national telephone survey of 700 parent-child pairs in the fall of 2007.

Here are some of the results:

• "85% of teens ages 12-17 engage at least occasionally in some form of electronic personal communication, which includes text messaging, sending email or instant messages, or posting comments on social networking sites." [They were twice as likely to send text messages as they were e-mail]

• "60% of teens do not think of these electronic texts as “writing.”"

• "50% of teens say they sometimes use informal writing styles instead of proper capitalization and punctuation in their school assignments"

• "38% say they have used text shortcuts in school work such as “LOL”"
  

• "25% have used emoticons (symbols like smiley faces ☺) in school work"
  

Wow. Color me wrong. I had no idea the practice was so rampant. But there may be a silver lining to this survey:

• "86% of teens believe good writing is important to success in life"

and they may know how to improve things:

• "82% of teens feel that additional in-class writing time would improve their writing abilities and 78% feel the same way about their teachers using computer-based writing tools"
  

So is this a big deal for recruitment and assessment? Potentially, if the work that needs to be done requires traditional, formal writing skill. But what if it doesn't? And what happens when this generation becomes managers--will they care if their subordinates use shortcuts and emoticons?

If one thing is made clear by this survey, it's not that teens don't think writing is important. It's that we may be in for a slight evolution in our written language. We may need to re-think the answer to this question: What is good writing?

Is age related to job performance?

Is there a relationship between age and job performance? It's an important question for many reasons, including the fact that claims of age discrimination appear to be on the rise. Ng and Feldman set out to better understand this issue and their meta-analysis is published in the March '08 issue of Journal of Applied Psychology.

Previous research has generally shown a weak relationship between age and job performance--at least when we look at objective measures. But the current authors set out to use a much broader array (10 to be exact) of criterion measures, including workplace aggression, safety performance, and OCBs.

So what did they find? Well, there's where things get a bit complex. Although there did not appear to be a relationship between age and several outcomes, including core task performance, creativity, and performance in training programs, it had stronger relationships with the other seven measures. In addition, age had a curvilinear relationship with core task performance and CWBs, and results varied depending on how the study was conducted.

So does age have a relationship with job performance? Like all important research questions, the answer is an emphatic it depends.


Other articles
There's quite a bit of good research in this volume, including:

- The development of a potentially useful way to predict team member performance

- A fascinating look at how frame-of-reference influences the validity of personality measures (pre-published version here)

- A discussion of the importance of the distinction between constructs (e.g., ability, personality) and methods (e.g., interviews) when comparing predictors in personnel selection

- How to test for adverse impact when your numbers are small (hint: significant testing bad, z-score good)

- Last but not least, a meta-analysis of the relationship direction between attitudes and job performance--what causes what? (hint: attitude matters...but not that much)

New MSPB study has gems

A new study out by the U.S. Merit Systems Protection Board (MSPB), "Attracting the next generation: A look at federal entry-level new hires," while focused on the federal government, has good information for all of us. The study of recent federal hires uses data from over 1,000 survey participants as well as personnel records. Some of the insights include:

- Both those under 30 and those 30+ were most likely to find out about the job through a friend or relative. On the other hand, those 30+ were more likely to use USAJOBs and agency web sites, whereas those under 30 (not surprisingly) were more likely to discover jobs through school job fairs or placement offices. In addition, those under 30 were more likely to use "non-government job sites."

- The most common forms of assessment used were (mostly unstructured) interviews and reviews of transcripts. Narrative statements, written exams, and job simulations were (unfortunately) used much less often--despite evidence of their superior ability to predict job performance.

- Agencies also relied heavily on the "point method" for reviewing training and experience--i.e., the more training/experience you have, the higher your score. This despite evidence that the correlation between experience and performance is complex and peaks fairly quickly, while education does not generally predict performance very well. Not to mention that they make little sense when hiring for entry-level jobs.

- There were very few differences between age groups when it came to what applicants wanted in a job. For both the under 30 and 30+ groups, salary and benefits came out tops. The only significant differences came when looking at pensions (more popular among those 30+) and health insurance and tuition reimbursement (more popular among those under 30). Job security was also mentioned by both groups as a big draw to the federal government.

- Those under 30 were much less likely to predict staying with the feds for their entire career (although still an impressive 45%), and more likely to predict eventually moving to the private sector.

- "Length of the process" was by far the biggest obstacle faced by applicants in their pursuit of a federal job.


MSPB recommends federal agencies make several improvements to their recruiting and assessment efforts, including:

- Use more predictive assessment tools instead of relying on training and experience measures.

- Use a balanced set of recruitment practices to reach all the various segments of the job market. MSPB points out that conducting a proper job analysis and identifying true minimum qualifications can help cut down the number of unqualified applicants.

- Market what is important to applicants. For federal agencies this means job security, benefits, and the ability to make a difference.

- Evaluate the hiring process and remove any obstacles that are unnecessarily lengthening the process.

- Avoid stereotyping based on generational assumptions, since the data indicate substantial similarities between age groups in what they want out of a job.

Again, good information for all of us. There's a lot more information in the report.