Unproctored internet testing: Safe for some tests?

One of the biggest trends in personnel assessment is the movement toward on-line testing. Many organizations are experimenting with so called unproctored Internet testing (UIT), where candidates are allowed to take the exams whenever, and wherever, they want.

Benefits? Extremely convenient for the candidate. Less administrative resources needed by the employer.

Costs? Bye-bye exam security, hello cheating opportunities. Not only is your test out for everyone to see, but you have no real way of knowing (sans biometric verification) who is taking the test.

Some organizations have decided the benefits outweigh the risks, and a new study in the June 2008 issue of the International Journal of Selection and Assessment may provide support for their position.

In it, the authors looked at over 800 applicants from nine European countries that took a test of perceptual speed in an unproctored setting, then followed this up with a proctored parallel version. Results? Not only was there no evidence of cheating, they found the opposite effect--people did better in the proctored setting.

Now before everyone throws out their proctored exams, note that this is a type of test that might be hard to cheat on--at least in one way. Because this is a perceptual speed test, there are no "right" answers that can be looked up. It also required very quick responses. So the only way to cheat would be to have someone take the test for you. Implication: it may make more sense to use certain UITs than others.

This topic is a source of much debate in the assessment community, and there is by no means consensus on the right way to go. But studies like this help!

Take a deep breath, because there's a lot more in this issue:

- The preliminary employment interview as a predictor of assessment center outcomes (fascinating look at how the AC may only make sense for mid-range interview scorers)

- A comparison of the common-item and random-groups equating designs using empirical data (for you IRT fans out there)

- The influence of external recruitment practices on job search practices across domestic labor markets: A comparison of the United States and China

- Beneath the surface: Uncovering the relationship between extraversion and organizational citizenship behavior through a facet approach (a more nuanced look at the relationship shows extraversion can predict OCBs)

- Comparing personality test formats and warnings: Effects on criterion-related validity and test-taker reactions (another good one...personality test added predictive validity beyond ability test but no validity difference between forced-choice and Likert scales, nor between warning and no-warning conditions; forced-choice and warnings may produce negative candidate reactions)

- Applicant selection expectations: Validating a multidimensional measure in the military (describes development of a new measure of applicant perception of the selection process)

- Selecting for creativity and innovation: The relationship between the innovation potential indicator and the team selection inventory

B = f (P,E)

One of the most famous axioms in social psychology is what's sometimes called "Lewin's equation" (after the famous psychologist Kurt Lewin): behavior is a function of both the person and the environment. This equation is good to keep in mind when looking at all kinds of human behavior, including recruitment and assessment.

Research presented in the May 2008 issue of Journal of Applied Social Psychology addresses this equation. Let's take a look at it and see if helps us answer an age-old question: What's more important--the observer or what's being observed?

Tell me if this situation sounds familiar. A hiring manager insists on hiring someone based on something they saw in the person's resume (e.g., the candidate graduated from a particular college), even though the person did not do well on a structured, validated assessment. The first study shows that HR is not immune to this phenomenon. In it, HR managers were presented with two types of information about a candidate: preliminary information (like a resume) and performance on an assessment center. The managers were then asked to rate the candidate. Results? Managers were unable to exclude the preliminary information, even though they had better information (the assessment center results) in front of them.

The second article looks at the legitimacy perceptions of promotion decisions and how they relate to information on deservedness (candidate performance) and entitlement (affirmative action). Participants felt that both deservedness and entitlement were related to legitimacy, but there was a gender effect--female participants felt increased resentment when the male candidate was promoted.

The third article is a fascinating take on how people how people perceive discrimination. Specifically, the authors looked at ambiguous situations and the impact of how "prototypical" the person doing the discriminating is. What they found was that the amount of control the perceiver felt they had over discrimination in their lives moderated the influence of the prototype effect. In other words, whether a white male (the prototype) was acting in a discriminatory fashion depended a great deal on the perceiver. Like research on stress, control was found here to have a significant effect on perceptions.

So given these three articles, what's more important--the observer or what's being observed? The research above gives us a clear answer, and one that validates the wisdom of Kurt Lewin: both.

Providing test information helps reactions

Providing applicants with pre-test information and preparation material may not help them pass, but it helps those that don't pass feel better about the process. These were the results of a study by Burns, Siers, and Christiansen reported in the March 2008 issue of the International Journal of Selection and Assessment.

Other articles include:

- An analysis of reactions to selection methods in the Netherlands. Interviews, work samples, and resumes came out on top.

- The development of a new measure of test anxiety, the Self- versus Other-referenced Anxiety Questionnaire.

- A study of validity transport statistics.

- A detailed comparison study of the FIRO-B, an assessment of interpersonal relations.

- A similar study of the PAPI-N, a personality measure.

- A study of how attitudes toward rating systems impact peer rating behavior.

- A look at how well multisource feedback on leadership competencies predicted subsequent interview performance (hint: pay attention to your supervisors and peers).

- Last but not least, a verification that "psychological hardiness" is an important facet related to success in high-stress occupations (in this case, U.S. Army Special Forces).

2007 Academy of Management Conference

There have been some news stories about one of the presentations at this year's Academy of Management (AOM) conference--about an online survey where a majority of respondents said that bad bosses either get promoted or have nothing happen to them. But there's a heck of a LOT of other good stuff at this year's conference. So take a deep breath and let's take a look...

First up, a whole set of presentations devoted to selection, including:

- Hiring for Retention and Performance
- Work Sample Test Ethnic Group Differences in Personnel Selection: A Meta-analysis
- Stigmatizing Effects of Race-Based Preferential Selection
- Longitudinal Changes in Testing Applicants and Labor Productivity Growth


Next, a session devoted to recruitment and selection, including:

- The Role of Sociolinguistic Cues in the Evaluation of Job Candidates
- Recruitment as Information Search: The Role of Need for Cognition in Employee Application Decisions
- A House Divided: Cooperative and Competitive Recruitment in Vital Industries
- The Practice of Sense-Making and Repair during Recruitment Interviews
- Overqualified Employees: Too Good to Hire or Too Good to Be True?


Next up, a session devoted to recruitment. Included topics:

- Customizing Web-Based Recruiting: Theoretical Development and Empirical Examination
- Network-based Recruiting and Applicant Attraction: Perspective from Employer and Applicants
- Fancy Job Titles: Effects on Applicants' Job Perceptions and Intentions to Apply
- Recruitment and National Culture: A Value-Based Model of Recruitment


Next, a set devoted to person-organization (P-O) fit, including:

- Going Beyond Current Conceptualizations of P-E Fit and Presenting a Status Report on the Literature
- Outcomes of Multidimensional Misfit: An Empirical Test of a Theoretical Model
- FIT: Scale Development and Initial Validation of a New Measure
- Considering the Contextualized Person: A Person-In-Content Approach to Goal Commitment


Next, a set on predictors of individual performance, including:

- An Examination of Ability-based Emotional Intelligence and Job Performance
- Predicting NFL Performance: The Role of Can-do and Will-do Factors
- A Fresh Perspective on Extraversion and Automobile Sales Success
- Auditor Effectiveness and Efficiency in Workpaper Review: The Impact of Regulatory Focus


Last but not least, one of my favorite topics, how organizations and individuals perceive selection. Topics include:

- Understanding Job Applicant Reactions: Test of Applicant Attribution Reaction Theory
- Effects of Ingratiation and Similarity on Judgments of P-O Fit, Hiring Recommendations and Job Offer
- The Effects of Resume Contents on Hiring Recommendations: The Roles of Recruiter Fit Perceptions
- Organization Personality Perceptions and Attraction: The Role of PO Fit and Recruitment Information


This is just a sample of what the conference has to offer; if you went, or otherwise know of other presentations we should know about, please share with us.

And no, most of the presentations aren't available on-line but the presenters' e-mail addresses are provided and most folks are willing to share.