Wednesday, February 18, 2009

Get in the game

Longtime readers know that I've considered one of the holy grails in our field to be a way of combining the interactivity and engaging content of video game technology (VGT) with recruitment and assessment. Yes, part of this is because I enjoy the occasional Nuka-Cola and killing the occasional troll, but there is so much potential in the marriage of these two fields that we can't ignore it.

Up until now, the best efforts have gone one of two ways. The first is creating an entire first person video game for recruitment purposes--this is what the U.S. Army did. The second is using VGT but in a very basic and limited way--this is what the FAA is doing. But to my knowledge no one has created a web-based tool that showcases the basic functionality of VGT while also serving as an assessment tool. In fact many people may not even know what this might look like.

Well I ran across something the other day (hat tip) that gets us pretty darn close. It's actually an onboarding program designed by Vestas, a Danish energy company. It takes the form of a situational judgment test (SJT) that leads new employees through an orientation of what Vestas does and their approach to their business.

I think once you've watched, you'll agree with me that the potential is vast.

So why this type of technology over, say, existing SJT solutions such as those offered by companies like Ergometrics and Biddle? Those definitely still have a place, and live actors are obviously higher fidelity, but here are some advantages to think about:

1. You can do more, and show more, with VGT. Need to show someone hanging onto the bottom of a helicopter then jumping to a rooftop? Not a problem, no wires required. Need to show someone underwater? Scaling a mountaineous peak? Again, much easier (and cheaper).

2. No screen actors required. No more worrying about makeup or getting the right shot--you create what you want. Of course voice talent is still very important if you decide to use sound.

3. It's just plain more modern. For folks that grew up watching cartoons and playing video games, they will naturally gravitate more toward something that feels familiar. Text job descriptions that link to an ATS? Yawn.

4. It will make you stand out. Yes, I know the unemployment rate is high here in the U.S., but don't think that means the end of competing for the most qualified. Now's the time to plan how you're going to compete when the pendulum swings the other way again.

5. It will stand the test of time. People still watch old cartoons. Very few old shows are on. That video shot of the desktop computer in the background may look outdated sooner than you'd like.

but perhaps most importantly:

6. VGT holds the promise of a truly interactive experience, where candidates explore their future work environment, make decisions, and learn about the organization. This has the potential to be both a realistic job preview that helps candidates decide whether to apply, as well as a measurement tool that gauges how well the candidate meets job requirements. (Yes this sounds a bit like Second Life but need not be so complex)

So what do we need to do moving forward? Here are some things we need to make this work:

1. More education. What do projects like this need to succeed? How much do they cost? What are the challenges and potential roadblocks?

2. Outreach to the VGT industry including the big companies (Activision Blizzard, EA, etc.) as well as the smaller shops, industry groups, schools, etc. No doubt they have much to teach us--but we have a lot to share as well. (As an aside, Activision has a very attractive Careers page that showcases some of their work, but they dump applicants right into their ATS like most companies--failed opportunity to continue the brand experience with a game-like character sheet!)

3. What are the psychometric implications? Is this just another version of unproctored Internet testing, or is there more here? How does this relate to run-of-the-mill adaptive testing? Are there demographic differences in willingness or performance?

Now what may throw a big monkey wrench into this is cost. Video games are not cheap (WOW cost $63M to develop). But we're not talking multi-user, latest video card, and all that stuff. This could be much shorter, more cartoonish, and much simpler.

I think this is the most exciting thing happening in assessment; I hope there are enough developers out there that agree.


Tom said...

There are at least a couple of major players in the online high fidelity selection game, with many more on the way. Shaker Consulting Group headed up br Dr. Brian Stern and Joe Murphy have created a facinating combination of sit judge, numerical reasoning, and 'day in the life' animations for Starbucks, and 40 other clients. They call them 'Virtual Job Tryouts'. Select International has something similar. They are pricy compared to more traditional testing on the startup costs but the Starbucks store manager VJT yielded amazing validity and utility in a study done in 2007. Tom Janz, Ph D, Global Human Asset Systems LLC. We have an online Performance Inventory(tm) authored by Dr. George Paajanen of PDI and Unicru fame. We also have an online, coached, Confirmed Achievements(tm) behavioral intervew that appears to have traction wuth the World Health Organization, among other places. I live in the Bay area and you can reach me at 415 690-6437 if interested.

BryanB said...

Thanks so much, Tom. Joe Murphy of Shaker CG is a friend and I've been impressed with what I've seen of the Virtual Job Tryout. I'm not familiar with the other tools you mention but I'd be very interested in seeing (as I'm sure other readers would) what they look like and whether they are truly interactive.

Joseph P. Murphy said...

As Brian notes, the multi-media nature of video gaming is what candidates expect from an on-line experience. For that to work in an assessment environment there must be a low or no threshold learning curve on mastering the technology interface. Most gamers get better over time. Multi-media assessment must deliver a fair and consistent first-time experience on a level playing field. Second Life for example, requires about 20 hours of practice to become a proficient navigator. Candidates will not tolerate, nor is it fair, for that degree of learning commitment to get into a job application.

Over 90% of candidates completing the Virtual Job Tryout® state they will refer friends and family to this job opportunity based on the nature of the application experience. When your candidate evaluation experience creates viral attraction, that is employment brand power at its finest. In today’s economy, the experience is the marketing.

While video-game like qualities are a worthy target to strive for, Web 2.0 principles provide a framework for fast and cost effective custom on-line candidate evaluation experiences. The Virtual Job Tryout® platform and methodology has been developed to deliver a job specific, company branded, highly engaging and informative candidate experience. We deliver custom development and validation at or below the cost of many off-the shelf solutions. We know this because of the number of clients that have chosen the Virtual Job Tryout® in competitively bid projects.

Tom is correct, (and thanks for the mention), the validity correlations achieved by the Virtual Job Tryout® demonstrate next-in-class capabilities of robust measurement methods. This is due in a large part to two factors: 1. The multi-method nature of the approach taken for candidate evaluation. 2. The proprietary techniques used to capture a comprehensive data set of subjective and objective measures of on-the-job performance. When it comes to quality validation analytics, size does matter.

It has been well documented that using two or more evaluation methods increases the power of prediction. It has also been well documented that work samples are very sound predictors of on-the-job performance. The Virtual Job Tryout® seamlessly combines four to six experiences comprised of job-specific work samples, job- relevant work history, and performance-appropriate work style. The customized-content nature of these experiences embeds realistic job preview within the evaluation. The candidate takes the job for a test drive. The recruiter gains a work sample that predicts performance. It is a double win for the recruiter and candidate.

Companies are justified in expecting more from their assessment technology. It can no longer be about “the test”. Companies that obtain the best hiring results view candidate evaluation as a measurement discipline for the business process called staffing. With this end in mind, the staffing process is evaluated by statistically exploring the inputs (candidates and candidate data) value-add components (evaluation, decision making and on-boarding) and yields (staffing waste, staffing rework, new hire performance variation).

Companies that view staffing as a business process know that calibration is at the core of process improvement. In the field of assessment validation = calibration. The closed loop analytics deployed with a Virtual Job Tryout® implementation feeds a periodic recalibration which hones in on the performance drivers. The Virtual Job Tryout® gets smarter over time, learning from experience. The staffing process becomes a data source which documents how each hiring decision is contributing to the business.

Brian you are correct about the investment and cycle time for developing a new game. The cost and time to market would have to plummet for even the companies with the greatest appetite for combining the “way cool” with the “way effective.”

There are more lead-time reasonable and cost effective methods for highly engaging and measurement rich candidate evaluation available today.

I too am available for any reader wishing to learn more.
Joseph P. Murphy
Shaker Consulting Group, Inc.
Developers of the Virtual Job Tryout®

BryanB said...

Thanks for your insightful comment, Joe. I look forward to opportunities in the future to see first hand what the VJT can do.