Gild.com: What the future looks like

The concept of a website that matches applicant skills to specific recruitments is...shall we say...not new.

What is new is a website that gets applicants to take actual assessments (of things like mathematical reasoning) so recruiters have a much better chance of finding the person that has the skills they need.

And it working.

This is the genius that is Gild, a website launched late last year devoted to "serious technologists" and currently being used for recruiting by companies like Oracle, eBay, and Salesforce.com. As of December they had over 100,000 users.

How do they do it? With a dash of good 'ol fashioned reinforcement, in the form of competitions for actual prizes that the target audience might like (like an iPad). The focus is on IT jobs, but one can easily envision this being expanded for other occupations (e.g., demonstrate your knowledge of multiple regression and win a free one-year subscription to SIOP!).

There are two main ways Gild gets users to take assessments: through certifications and competitions. Certifications are short multiple-choice tests designed to measure proficiency in things like ASP.NET, SharePoint, and Unix (and some more general competencies like English proficiency). They're easy to take and (at least if my middling knowledge of IT is any indication) difficult to fake. They've even incorporated reinforcement into adding members (invite friends for a chance to win an iPad!).

Competitions are where things get really interesting. They're also short m-c tests, and here are some examples of competitions under way as of this writing:

- PHP Elite (prize: Kindle)
- Java Elite (prize: iPad)
- Mathematical Reasoning (prize: AppleTV)

It's the social competition that seems to be the key. The group interaction extends even to their excellent support forum. For example I see that someone suggested practice exams or sample questions, and the site was quick to praise the idea and promise to investigate.

This website demonstrates that it is possible to get internet applicants to complete real assessments as part of an online profile--when properly motivated. Yes, I know, unproctored testing is subject to faking, blah, blah, blah. I just don't buy that argument anymore. Use confirmatory testing; end of story.

One thing I noticed, which may not be that surprising: the leaderboard is currently dominated by folks (okay, almost all men) from India. I mean...wiping the floor with the other countries. Now they have offices in India (and China, and the U.S.), so maybe it's better known there. Or maybe their focus on technology is showing dividends.

Another interesting...feature...is that the site is supposed to be exclusively for direct employers, not third-party recruiters. No trolling here. Once you create an account, you have the ability to post jobs (or "job cards"), create competitions, and manage your company profile. Interestingly, the posting process allows you to specifically select three skills--and their level--to target. Like a mini-mini job analysis. It will even forecast the supply and demand dynamically based on your requirements. A one-month "silver" posting is free but is smaller than the $50USD-"gold" posting that also includes 100 invites and better placement. Still, very affordable (I mean, Dice is $500).

Behind all this is the Professional Aptitude Council (PAC), a company that creates certifications and technologies to deliver them. The website states that their mission is in large part to ensure that talented individuals get opportunities based on their merit.

Gild is a great example of how to use technology to engage applicants, create more legitimate profiles, and offer employers a more accurate method to match individuals to specific recruitments.

Hat tip. You can read a little more about the company's history and purpose here.

We can make assessments "fun"...should we?

Remember when tests were fun?

Neither do I. Tests and assessments have a long history of being about as popular as the dentist. Starting in grade school many come to dread them as lifeless--and often inaccurate--judges of worth. (Of course doing well on them that tends to improve your view)

Tests don't have to be boring. We write structured interview questions and multiple-choice questions because that's what we've always done. And we know how to do it right.

But there are plenty of ways of making them more interesting, from the way they're written (try: "You are in a maze of twisty passages, all alike"), to their presentation (e.g., animation, video), to the way people progress (e.g., adaptive testing), to the way results are given to you ("You've got the high score!"). Today more than ever before we have the flexibility to take those dry, monochromatic presentations and turn them into something eye-catching and even...dare I say...fun?

(10 bonus points to those of you who caught the Adventure reference in the preceding paragraph)

The question is: Should we?

There's been quite a bit written lately about the "gamification" of assessments. Heck, I've been on that bandwagon for years. America's Army, first published in 2002 as the U.S. Army's foray into first-person shooters, was an early example of the potential marriage between staffing and entertainment--yes it's not technically a personnel assessment but the recruiting mission is obvious as is the potential use of the results. Since then we've seen a steady stream of innovation, from the use of branching video to realistic job preview-type assessments presented online.

There are several reasons why we might want to make assessments a wee bit more entertaining:

A) Because we can. Please don't use this reason.

B) As a recruiting tool to help you distinguish yourself from competitors ("Look, we're fun and cool! Join us!")

C) To encourage candidates to complete the assessment ("Yes it's a little long, but the time will fly by!")

D) As part of a realistic job preview ("Not sure if you want the job? Find out virtually!"). Nothing wrong with that. Self-selection out is good.

E) Because it helps us measure more accurately. Ah-ha. Now we're getting somewhere. To the extent that entertainment/interactivity helps us overcome candidate fatigue or other error, or otherwise helps us measure relevant KSAPs more accurately, we've won the game.

(10 bonus points to those of you who saw noticed the irony of me using a traditional multiple-choice presentation in that last part)

(50 bonus points to those of you who have noticed the use of bonus points in this post)

So we have a number of reasons we might want to make assessments more fun. But there are some reasons why we might not want to. Or at least that should make us pause:

1) Tests are serious. No, really. They have an enormous impact on people's life. We don't want to water down their nature so much that we disrespect our applicants.

2) The "tests" out there right now that are the most "fun" are also not ones you'd want to use to select people (e.g., which animal are you?), although there are some surprising hybrids (find your Star Wars twin).*

3) Those assessments that are kinda-pitched-as-actual-assessments-but-not-really-don't-hold-us-to-that, have already started blurring the line (e.g., True Colors). We want to draw a clear distinction between assessments purely for fun and "okay, you need to take this seriously, it's for a job."

4) We can easily mess this up. All it takes is a high-level manager getting into his/her head that we need to "make these test things more fun" and suddenly we're pressured into creating an expensive mess that doesn't deliver (like Daikatana).

(10 bonus points to those of you who noticed I switched response options from letters to numbers)

(50 bonus points to those of you that completed all three of the example assessments)

(100 bonus points if you know what Daikatana is)


So is there room for us to be a little more creative and investigate alternate--more immersive, interactive--ways of assessing candidate qualifications? Absolutely. But should we use caution to make sure we don't have a big-budget flop on our hands? You bet.


Now count up your points from this blog post. How did you do?

0 points: Wait...you did READ this, right?

10-20 points: Okay, maybe you're tired.

20-130 points: The force is strong with you...but you are not a Jedi, yet.

More than 130 points: Call me.


* I'm an owl. Or maybe a penguin. Oh, and kinda like Darth Vader. But also Princess Leia. And Mon Mothma. Now I'm confused.

Q&A with Piers Steel: Part 2

Last time I posted the first part of my Q&A with Piers Steel, co-author of a recent piece in Industrial and Organizational Psychology (that I wrote about here) on synthetic validity and a fascinating proposition to create a system that would greatly benefit both employers and candidates. Read on for the conclusion.

Q4) Describe the system/product--what does it look like? For applicants? Employers? Governments?

A4) How do we do it? Well, that’s what our focal article in Perspectives on Science and Practice was about. Essentially, we break overall performance into the smallest piece people can reliably discern, like people, data, things (note: our ability to do this got some push back from one reviewer – that is, he was arguing we can’t tell the difference if people are good at math but not good at sales and vice-versa – it is a viewpoint that became popular because researchers assumed that “if it ain’t trait, it’s error”). We get a good job analysis tool that assesses every relevant aspect of the job, such as job complexity. We get a good performance battery, naturally including GMA and personality. We then have lots of people in about 300 different jobs take the performance battery, have their performance on every dimension as well as overall assessed to a gold standard (i.e., train those managers!), and have their jobs analyzed with equal care with that job analysis tool. From that, we can create validity coefficients for any new job simply by running the math. It is basically like validity generalization plus a moderator search, where once we know the work, we can figure out the worker. Again, read the article for more details, but this was basically it.

Once built all employers need to do to get a top-notch quality selection is describe their job using the job analysis tool and then as fast as electrons fly through a CPU, you get your selection system, essentially instantly. It is several orders of magnitude better in almost every way from what we have now from almost every criterion.

Q5) What are the benefits--to candidates, employers, and society?

A5) Everyone had a friend who struggled through life before finding out what they should have been doing in the first place. Or changed a job for a new company only to find they hated it there. Or never found anything they truly excelled at and just tried to live their lives through recreational activities. Everyone has experienced lousy service or botched jobs because the employee wasn’t in a profession that they were capable of excelling in. Everyone has heard of talented people who were down and out because no one recognized how good they really were.

Synthetic validity is all about making this happen less. How much less is the real question. If we match people to jobs and jobs to people wonderfully now, then perhaps not at all. But of course, we know that presently it's pretty terrible.

Now synthetic validity won’t be able to predict people’s work future perfectly, but it will do a damn sight better job than what we have now. Also, the best thing about synthetic validity is that it is going to start off good and then get better every year. Because it is a consolidated system, incremental improvements, “technical economies,” are cost effective to pursue and once discovered and developed, they are locked in every time synthetic validity is used.

Right now, we have a system that can only detect the largest and most obvious of predictors (e.g., GMA) because of sample size issues, but can’t pursue other incremental predictors because they aren’t cost effective for just one job site. By the very nature of selection today, we are never going to get much better. As I mentioned, nothing major has changed in 50 years and nothing major will change in the next 50 if we continue with the same methodology. Synthetic validity is a way forward. With synthetic validity, the costs are dispersed across all users, potentially tens of millions, making every inch of progress matter.

So, what we will get? Higher productivity. If synthetic validity results in just a few thousand dollars of extra productivity per employee each year, multiply that by 130 million, the US work force. Take a second to work out the number – it’s a big number.

Also, people should be happier in their jobs too, creating greater life satisfaction. They will stay in their jobs longer, creating real value and expertise. Similarly, unemployment will go down as people more rapidly find new work appropriate to their skills. In fact, I can only think of one group that won’t like this – the truly bad performer. They are the only group that wouldn’t want better selection.

Q6) Finally, what do you need to move forward?

A6) So far, no one I know is doing this. There are some organizations who think they are doing synthetic validity, though it is really just transportability and they aren’t interested in pursuing the real thing. Partly, I think it is because the real decision makers don’t know about synthetic validity or don’t understand it. I could do more to communicate synthetic validity, though I have done quite a bit already. I have sent a few press releases, received a dozen or two newspaper interviews (Google it), contacted a few government officials on both sides our border, and pursued a dozen or so private organizations. Part of my reason to do this interview here is to try to get the word out. So far, all I got back was a few “interesting” but no actual action.

I used to think this lack of pursuit was because synthetic validity was so hard to build, requiring 30,000 people -- but we know a lot more now. In the Perspectives article, McCloy pointed out we could allow ourselves to use subject matter experts to estimate some of the relationships. That won’t be as good as if we gathered the data ourselves, but we could get something running real quick, though later we would upgrade with empirical figures. Consequently, the reason why this isn’t built isn’t because it is too difficult. Also, the payoff would eventually be cornering the worldwide selection and vocational counseling market. I am not sure what that is worth but I imagine you could buy Facebook with change left over for MySpace if you wanted to. The value of it then isn’t the problem either.

I am coming to the conclusion that despite the evidence, to most people it is just my word as an individual. I’m a good scientist, winner of the Killam award for best professor at my entire University, but it still isn’t enough. You need the backing of a professional association and so far ours [SIOP] hasn’t yet taken a stand. As a professional organization, we should be promoting this, using the full resources of our association. I admit that I am “a true believer,” but this seems to be one of the bigger breakthroughs in all the social sciences in the last 100 years. Alternatively to the backing of a professional association, we need a groundswell where hundreds of voices repeat the message. I will do my bit but hopefully I will have a lot of company.

If you think I am overstating the case regarding synthetic validity, show me where I’m wrong. We handled all the technical critiques and issues in the Perspectives article. Right now, you have to make the argument that “human capital” doesn’t matter, that being good or bad at your work doesn’t matter. And if you try to make that case, I don’t think you are the type of person who would be even worth arguing with.


I'd like to thank Dr. Steel for his time and energy. I truly hope this idea sees the light of day. If you are interested in moving this forward, leave a comment and I can put you in touch with him.

Q&A with Piers Steel: Part 1

A few weeks ago I wrote about a research article that I think proposes a revolutionary idea: The creation of synthetic validity database that would generate ready-made selection systems that would rival or exceed the results generated through a traditional criterion validation study.

I had the opportunity to connect with one of the articles authors, Piers Steel, Associate Professor of Human Resources and Organizational Dynamics at the University of Calgary. Piers is passionate about the proposal and believes strongly that the science of selection has reached a ceiling. I wanted to dig deeper and get some details, so I posed some follow-up questions to him. Read on for the first set of questions, and I'll post the rest next time:

Q1) What is the typical state of today's selection system--what do we do well, and what don't we?

A1) Here is quote from well a respected selection journal, Personnel Psychology: “Psychological services are being offered for sale in all parts of the United States. Some of these are bona fide services by competent, well-trained people. Others are marketing nothing but glittering generalities having no practical value.... The old Roman saying runs, Caveat emptor--let the buyer beware. This holds for personnel testing devices, especially as regard to personality tests.”

Care to try and date it? It is from the article “The Gullibility of Personnel Managers,” published in 1958. Did you guess a different date? You might of, as the observation is as relevant today as yesterday -- nothing fundamental has changed. Just compare that with this more recent 2005 excerpt from HR Magazine, Personality Counts: “Personality has a long, rich tradition in business assessment,” says David Pfenninger, CEO of the Performance Assessment Network Inc. “It’s safe, logical and time-honored. But there has been a proliferation of pseudo tests on the market: Caveat emptor.”

Selection is typically terrible with good being the exception. The biggest reason is that top-notch selection systems are financially viable only for large companies with a high-volume position. Large companies can justify the $75,000 cost and months to develop and validate and perhaps, if they are lucky, have the in-house expertise to identify a good product. Most other employers don’t the skill to differentiate the good from the bad as both look the same when confronted with nearly identical glossy brochures and slick websites. And then the majority of hires are done with a regular unstructured job interview – it is the only thing employers have the time and resources to implement. Interviews alone are better than nothing but not much better – candidates are typically better at deceiving the interviewer than the interviewer is at revealing the candidate.

The system we have right now can’t even be described as being broken. That implies it once worked or could be fixed. Though ideally we could do good selection, typically, it is next to useless, right up there with graphology, which about a fifth of professional recruiters still use during their selection process. For example, Nick Corcodilos reviews how effective internet job sites are getting people a position. He asks us to consider “is it a fraud?”

Q2) What's keeping us from getting better?

A2) Well, there are a lot of things. First, sales and marketing works, even if the product doesn’t. When you have a technical product and untechnical employer or HR office, you have a lot of room for abuse. I keep hearing calls for more education and that management should care more. You are right they should care more and know more. People should also care and know more about their retirement funds as well. Neither is going to change much.

Second, the unstructured job interview has a lot of “truthiness” to it. Every professional selection expert I know includes a job interview component to the process even when it doesn’t do much, as the employer simply won’t accept the results of the selection system without it. There are some cases where people “have the touch” and are value added but this is the exception. Still, everyone thinks they are gifted, discerning, and thorough. This is the classic competition between clinical and statistical prediction, with evidence massively favoring the superiority of the latter over the former but people still preferring the former over the latter (here are few cites to show I’m not lying, as if you are like everyone else, you won’t believe me: Grove, 2005; Kuncel, Klieger, Connelly, & Ones, 2008).

Third, it just costs too much and takes too much time to do it right. Also, most jobs aren’t really large enough to do any criterion validation.

Q3) What might the future look like if we used the promise of synthetic validity?

A3) Well, to quote an article John Kammeyer-Mueller and I wrote, our selection systems would be "inexpensive, fast, high-quality, legally defensible, and easily administered.” Furthermore, every year they would noticeable improve, just like computers and cars. A person would have their profile taken and updated whenever they want, with initial assessments done online and more involved ones conducted in assessment centers. Once they have the profile, they would get a list of jobs they would likely be good at, ones that they would be likely good at and enjoy, and ones they would be likely good at, enjoy and that are in demand.

Furthermore, using the magic of person-organization fit, you inform them what type of organization they would like to work for. If someone submitted their profile to a job database, every day job positions would come to them automatically, with the likelihood of them succeeding at it. These jobs would come in their morning email if they wanted it. Organizations would also automatically receive appropriate job applicants and a ready built selection system to confirm that the profile submitted by the applicant was accurate.

Essentially, we would efficiently match people to jobs and jobs to people. I would recommend people update their profile as they get older or go through a major life change to improve the accuracy of the system, but even initially it would be far more accurate than anything available today -- a true game changer.

Follow-up: Some might see a contradiction here. You cite an article that bashes internet-based job matching, yet this is what you're suggesting. Would your system be more effective or simply supplement traditional recruiting methods (e.g., referrals)?

A: Yup, we can do better. The internet is just a delivery mechanism and no matter how high-speed and video enabled, it is just delivering the same crap. This would provide any attempt to match people to jobs or jobs to people with the highest possible predictiveness.


Next time: Q&A Part 2

References:
Grove, W. M. (2005). Clinical versus statistical prediction: The contribution of Paul E. Meehl. Journal of Clinical Psychology, 61(10), 1233-1243. doi: 10.1002/jclp.20179

Kuncel, N. R., Klieger, D., Connelly, B., & Ones, D. S. (2008, April). Mechanical versus clinical data combination in I/O psychology. In I. H. Kwaske (Chair), Individual Assessment: Does the research support the practice? Symposium conducted at the annual meeting of the Society for Industrial and Organizational Psychology, San Francisco, CA.

Stagner, R. (1958). The Gullibility of Personnel Managers. Personnel Psychology, 11(3), 347-352.

ClicFlic offers assessment innovation

A while back I posted about a creative use of technology that Vestas was using for onboarding. At the time I wrote about the potential I saw for the use of such technology for assessment, but actually creating these videos was a bit of a mystery from the customer side. Now I've come across a vendor that allows us to create these tools.

I don't post about specific products very often--usually I focus on research and best practices--but I have made occasional exceptions. When I see a product that I think has the potential to be innovative, highly effective, and highly valid, I want to share the wealth.

Such is the case with ClicFlic. In a nutshell, ClicFlic allows customers to create customized interactive web-based videos that can be used for things like situational judgment tests (SJTs). But we've seen that before, right? What I hadn't seen was the branching ability of ClicFlic.

Historically, video-based testing, whether Internet-enabled or not, presents all candidates with the same content. A situation is presented, and the candidate is provided with either several pre-determined responses or an open-ended response area. But much like traditional computerized adaptive testing (CAT), ClicFlic allows for the creation of branching videos. In other words, what the user sees in the next segment will vary depending on how they respond on the current one.

Although most of the examples you'll see on their website involve customer service or training applications, the technology is easily adaptable to assessment situations, as you can see from this example.

I had an opportunity to speak with Mike Russiello, President and CEO of ClicFlic (and co-founder of Brainbench) and he allowed me to peek "under the hood"--what I saw looked plug-and-play easy. The scripting branches are easy to generate, videos simple to upload, and you can quickly assign points to different responses. The videos are flash-based and you can easily generate the HTML to place it on a webpage.

Want to learn more? Check out the examples on their website--the demo on the front page will give you a good feel for the technology. Here are some others that will give you an idea of the possibilities. For assessment-specific usages, here you can select several different types of items with some characters you may recognize.

Questions? You can learn more about how the tools are built here. You may also run into Mike at SIOP if you have questions. Finally, they're also planning on an upcoming webcast through tmgov.org.

I hope this sparks some interest for you and maybe even some ideas about where this technology could be taken even further (RJPs anyone?).

Hybrid tests and the June '09 IJSA

What do you get when you combine a structured interview with a performance assessment? Perhaps some sort of hybrid with pieces from both sides. In the June 2009 issue of the International Journal of Selection and Assessment we find out more.

Morgeson et al. describe the development of a "performance interview" that combines a structured interview with an on-site performance demonstration. Essentially this involved going to the relevant work area (this study was for parts manufacturers) and asking a series of questions to determine promotability, such as "How do you set up this machine?" It's fascinating stuff, and it worked (using concurrent measures), although it might be challenging to use for less observable performance measures. For more details, check out the in press version here; the recipe book starts on page 12, and check out the example on page 41.

What else is in the issue? Take a look:

Predicting managerial readiness in Chinese workers

Is inflation in personality inventories necessarily a bad thing?

Do occupations tend to have their own personality? (yep)

Leadership effectiveness: Self- versus other-ratings (check out who tends to inflate)

CWBs: The organization plays a role

Biodata continues to shine (this time in healthcare organizations)

Is handwriting analysis popular among European employers? Not so much.

Can you predict military performance using personality inventories? Seems so.

Job experience v. personality measures in a small sample

HRE announces best HR ideas for 2009

Recently Human Resource Executive posted their Best HR Ideas for 2009 winners. Among the topics (benefits, HR tech, talent management, and T&D) they highlight some pretty interesting examples related to recruitment and assessment. Here's a sample:

- AlliedBarton Security Services gathered productivity data and discovered many managers were leaving at the peak of their productivity; this allowed them to focus additional resources on targeted retention efforts

- US Cellular uses web-based meeting technology to allow hiring supervisors to communicate about their candidates, ensuring more consistency and enhancing communication

- Proctor and Gamble, working with DDI, moved their cognitive ability testing online and made them adaptive

- SITA developed a leadership development portal where employees can literally "gauge" their competencies and receive targeted instruction

- BPO firm iQor created an online test designed to measure "emotional, intellectual, and conversational skills", reducing the need for interviews, increasing productivity, and reducing turnover

Feedback can Rypple

Every once in a while I post about something that's not directly related to recruitment and selection. Particularly if it's about a cool new technology. And when that cool new technology is free, well heck...I can't help myself.

Which brings me to Rypple. Rypple is a website that allows anyone to request anonymous feedback from people they work with (or, frankly, anyone). It's similar to Checkster (which also has a free feedback system called "Talent Checkup" with limited functionality) but has a different focus. Whereas Checkster has tools for both employers and individuals, Rypple is focused on the individual. The interface is also a little different, a little more micro-bloggy.

Why do I think this is important? Because we know that positive reinforcement is the best way to encourage the same behavior, yet many employees feel under-appreciated and lack the information they need to improve their performance and gain insight. Supervisors and managers are often unavailable or distracted, forgetting how important feedback is. Tools like Rypple put the power (or responsibility) with the employee.

You can easily see how this could become part of not only performance management but things like leadership development and team success measurement.

One last note: they are in beta, so the site and services may change. You can read about some of the developments on their blog.

Hat tip

Looking far and wide

When it comes to finding talented individuals, how far and wide do you look? Are you as creative as you could be?

In a recent article James E. Challenger, of the outsourcing firm Challenger, Gray, and Christmas, described the results of a new study in which half of the 100 HR executives polled stated their companies work informally with former employees; 23% considered stay-at-home parents to be valuable recruiting targets. The goal? Finding folks that are experienced and easy to train.

What does your organization do when people leave? Does it go beyond getting a forwarding address? How about following a structured approach to keep track of talented individuals in case their next job fizzles?

Challenger cites Lehman Brothers as a leader in this area with its Encore program, which according to the website is "designed to facilitate networking and professional development opportunities for individuals interested in resuming their careers in the financial services industry. Ideal candidates are women and men, preferably with industry-related experience at a vice-president level or above, who have voluntarily left the workforce for a year or more."

Does your organization actively recruit people that have been out of the workforce for a year or more? Or are these people seen as "stale"?

The article also includes "resources for returning parents", including:

UCEAdirectory.org (searchable database of continuing education courses)

Meetup.com (real-world social networking)

Modernmom.com (advice on activities and work-life balance)

Showmomthemoney.com (money tips, degree links, and more)

Ladies Who Launch (networking and entrepreneurial advice)

Has your organization considered recruiting efforts that target these types of groups? Or is it hoping that qualified applicants find you?

Some things to think about as we all work on being more creative with reaching out to all qualified candidates. I bet there are a lot of folks out there that would love to see a list of employers willing to hire returning workers (as well those that are open to part-time arrangements).

Checkster and SkillSurvey automate reference checking

When it comes to things that supervisors (and, frankly, HR) don't look forward to, reference checking probably ranks in the top 5. Checking references is time consuming and difficult to do well, because many references refuse to do more than confirm name, job title, salary, and employment dates for fear of getting sued. This is unfortunate, since lawsuits in this area are quite rare and reference checks can be a great source of information.

So it was with no small amount of excitement that I discovered two web-based services that are automating the reference check process--Checkster and SkillSurvey. The basic idea behind these services is that a brief survey consisting of both rating scales and open-ended questions is sent out electronically to references; responses to these surveys (generally at least 3) are combined by the services into an overall report for the employer. While the services are substantially similar, in this post I'll give you a brief overview of each.

Checkster

Checkster is the brainchild of CEO Yves Lermusi, formerly of Taleo (and frequent contributor to ERE). Lermusi noticed that the frequency and quality of the performance feedback most people receive drops dramatically when they move from school to work, making it difficult for people to understand their strengths and areas for development. To help remedy this, he developed Checkster to be a "personal feedback management tool"--a focus that he says distinguishes it from other services, whose bread and butter is employer-based reference checking. Applicants receive the results of the reference check, just as the employer does, with the idea that this information will be used to help people develop and make better decisions regarding their career.

With Checkster, the employer simply enters the name and e-mail address of each applicant along with the requisition and selects the type of survey to be delivered (Checkster also has a 360-degree survey). That's it. Simple, eh? The applicant takes it from there, logging into Checkster and entering reference names and contact information. References have 7 days to take the quick and confidential survey, and Checkster compiles the resulting information into a report after at least three responses have been collected. From the employer's side, a simple account screen allows you to manage your requisitions and see the status of each. You can see an overview of how it works here, and watch a demo here that includes pictures of a report.

Big bonus: Checkster also has a free employment verification feature which will send an e-mail to previous employers to verify dates of employment, reporting structure, compensation, and eligibility for rehire.

Price: $50 per requisition, which allows you to check references for up to five candidates with a maximum of 15 references per candidate (volume discounts are also available).

SkillSurvey

As I mentioned, SkillSurvey and Checkster work in a similar fashion--the employer enters candidate information, the candidate enters reference information (or the employer can), references evaluate the candidate, and SkillSurvey generates the report.

Differences between Checkster and SkillSurvey that I observed:

1) SkillSurvey allows you to choose different types of surveys depending on the job. Each includes competencies developed by SkillSurvey staff. For example, there are different surveys for sales positions, IT positions, and HR positions (click here for an example for Marketing Manager).

2) Each point on SkillSurvey's rating scale is anchored, which could potentially lead to better reliability.

3) SkillSurvey reports are not automatically available to the candidate (unlike Checkster)--this reflects the emphasis that SkillSurvey places on being primarily a tool for the employer, versus Checkster's focus on individual development.

4) SkillSurvey has a sourcing component built in--you can download a spreadsheet that contains all the information on reference-givers that you can sort and use to identify applicants (very cool).

5) Checkster's reports are a little shorter and more graphical, while SkillSurvey reports are more text-based and extensive.

6) In terms of customization, SkillSurvey offers many options for altering things like turnaround time, and even weighting questions.

7) The actual text that goes out from candidates is easier to modify using Checkster.

A SkillSurvey overview video with screens of surveys and reports is available here, and sample reports are here. They even have a blog written by Doug LaPasta, their founder and chairman.

Price: $59 for one candidate with significant discounts for volume; usually charged in units of 100 candidates. The employer controls the number of reference givers required for completion of the report (anywhere from 2-15).

By the way, SkillSurvey was selected as a Top HR Product for 2007 by Human Resource Executive.

Summary

Both products have the potential to dramatically decrease the amount of time spent checking references, and have the added benefits of standardization as well as indicating an affinity for technology. Both companies have taken steps to ensure reference givers feel comfortable giving out information. The information may also be of higher quality since the process is being handled by a third party.

Both services were extremely easy to use. I found representatives from both companies to be knowledgeable and helpful. I'm sure as both products mature we'll see great additions, including hopefully an increased ability to gather off-list checks and even more options for tailoring the surveys.

Some things to keep in mind: (1) Like all mass-mailing type services, make sure e-mails from these companies don't get blocked by firewalls; and (2) Because some candidates may provide false references, do periodic spot checks (e.g., by verifying name & e-mail address).

I hope this has peaked your interest; I suggest checking out both products to see if either would make your life easier!

OPM Introduces Assessment Decision Tool

The U.S. Office of Personnel Management (OPM) continues to innovate in the area of web-based assessment offerings with the introduction of the Assessment Decision Tool.

This interactive application allows you to enter competencies required for a particular position for seven occupational groups, including:

- Clerical/technical
- Human resources
- IT
- Leadership occupations
- Professional/administrative
- Science/engineering
- Trades/labor

After entering in this information, the system will create a matrix that identifies which assessment tools would be the best fit for each competency, then gives you totals so you can see which methods might be best overall. These matches were identified by a panel of OPM psychologists who specialize in personnel assessment.

The final step is the generation of a comprehensive report that summarizes the position competencies, assessment matches, and information about each assessment method. There is also a reference section for those wishing to investigate further. The report can be downloaded in HTML format and OPM is working on PDF functionality as well.

A great idea that automates competency-exam linkages that many assessment professionals do routinely. Now if they can just have the system create the actual tests...

More games

I've posted before (here and here) about how Google and other companies are literally using boardgames as part of their applicant screening process, and how I'm not a big fan of this technique.

The September, 2007 issue of Business 2.0 has an article titled "Job Interview Brainteasers" that highlights another type of game employers play--this time, it's asking "creative" questions during the interview.

Let's take a look at some interview questions from the article and who's asked them:

How much does a 747 weigh? (Microsoft)

Why are manhole covers round and not, say, square? (Microsoft)

How many gas stations are there in the United States? (Amazon.com)

How much would you charge for washing all the windows in Seattle? (Amazon.com)

You have 5 pirates, ranked from 5 to 1 in descending order. The top pirate has the right to propose how 100 gold coins should be divided among them. But the others get to vote on his plan, and if fewer than half agree with him, he gets killed. How should he allocate the gold in order to maximize his share but live to enjoy it? (eBay, and, similarly, Pirate Master)

You are shrunk to the height of a nickel and your mass is proportionally reduced so as to maintain your original density. You are then thrown into an empty glass blender. The blades will start moving in 60 seconds. What do you do? (Google)

These questions have been around for quite a while and are used to measure things like creativity and estimation ability. The question is: Are they any better than board games? Probably. But they're still a bad idea.

Why do I say that? Well, first of all, a lot of people find these questions plain silly. And this says something about your organization. Sure, some people think they're fun or different. But many more will scratch their head and wonder what you're thinking. And then they'll wonder if they really want to work with you. Particularly folks with a lot of experience who aren't into playing games--they want to have a serious conversation.

Second, there are simply better ways of assessing people. If you want to know how creative someone is, ask them a question that actually mirrors the job they're applying for.

Want to know how they would tackle a programming question? Ask them. In fact, you can combine assessment with recruitment, as Spock recently did.

Want them to estimate something? Think about what they'll actually be estimating on the job and ask them that question. And so on...

Another advantage of these types of questions? The answers give you information you can actually use. (Hey, you've got them in front of you, why not use their brains)

If you don't really care about the assessment side of things, and in reality are just using these questions as a way to communicate "we're cool and different" (as I suspect many of these companies are doing) there are better ways of doing this. Like communicating in interesting and personal ways (e.g., having the CEO/Director call the person). Like talking about exciting projects on the horizon. Like asking candidates what THEY think of the recruitment and assessment process (gasp!).

My advice? Treat candidates with respect and try your darnedest to make the entire recruitment and assessment process easy, informative, and as painless as possible. Now THAT'S cool and different.

B-school requires PPT slides for admission

So apparently Chicago's Graduate School of Business is going to require four pages of PowerPoint-like slides as part of its admission process this fall.

According to school reps, this will allow students to "show off a creative side that might not reveal itself in test scores, recommendations and even essays." Another rationale given by the school is that students will have to master this type of software before entering the business world.

One problem I see here is the vast majority of applicants will already know PowerPoint--if you get through high school and college without using it, I'm betting you're the rare applicant.

The larger problem here is the same problem employers face with supplemental questionnaires and work samples--namely, who did it? In high-stakes situations like school admissions and job applications, people are known to take, shall we say, less than ethical routes to increase their chances.

The benefit of something like GPA or the GMAT is identity verification--you can be virtually assured (as long as you can validate the numbers) that the person who's applying is the one that took that test.

With things like previous work samples, resumes, and now this PowerPoint idea, you have no idea who actually created the product. So you make an admissions or hiring decision based on an assumption. Do you validate that they actually created these documents? Probably not. Even if you wanted to, how would you do it?

It might not even matter, since this may be more of a way to add excitement to application reviews and to simply get more applicants, which the school acknowledges. There seems to be a trend among organizations to implement projects that aren't so much concerned with valid predictions of performance but with simply attracting attention. This will likely get even more blatant as organizations struggle to keep their staffing levels up in the coming years.

But we should keep this in mind: gimmicks may attract some applicants, but do they turn others away? What about highly qualified individuals who think, "Well that's silly." That's why the best solutions will pique interest while being as close as possible to the actual job (or school) environment. How about asking applicants to give a presentation as part of their interview--now that's a skill they'll need. Plus, absent any Mission Impossible-style disguises, you can be pretty sure the person in front of you is who they claim to be.

Legos: They're not just for Google anymore

So apparently Legos (or "Lego bricks") are enjoying quite the popularity among corporate recruiters these days.

Not only did Google use them at Google Games (and apparently employees enjoy them as well), PricewaterhouseCoopers (PwC) asks candidates to create a tower with Legos, according to this Economist article.

So what exactly are candidates doing? PwC asks candidates to create the tallest, sturdiest structure they can using the fewest number of "bricks." Google asked candidates to build the strongest bridges they could.

Is this a valid form of assessment? A "professional Lego consultant" in Buenos Aires stated that, "Lego workshops are effective because child-like play is a form of instinctive behaviour not regulated by conscious thought. " There's even a website devoted to Lego's efforts in this area--Serious Play.

So my question is: Do most of us do work that is "not regulated by conscious thought"? Perhaps sometimes, say in emergencies. But the vast majority of time we're pretty darn deliberate in our actions. The only situation I can see where this might be predictive of actual job performance would be for jobs like bridge engineer or architect. But...computer programmer? If I wanted to know how creative a programmer is, I'd ask him/her to solve a difficult coding problem.

Does this even matter? Perhaps not (unless they're sued). As one of the candidates states, correctly I think, "It was as much advertising as a way of trying to get recruits." So in this day and age of "talent wars", this may be just another branding technique.

Will it be successful? Probably depends on how much the candidate likes to play with blocks.

This post is dedicated to my Grandpa Ben, who had a great sense of humor. And probably would have thought using Legos in this way is a bit silly :)

TalentSpring takes on peer ratings

A while back I mentioned RecruitmentRevolution, a UK site focused on temporary employment that allows previous employers to input reference scores for use by future employers. Creative idea, if ya ask me. I've often wondered if someday there will be a general database of verified work history that employers could easily check.

Now along comes TalentSpring with a similar idea. This time it's not previous employers, it's peers. TalentSpring uses something it calls a "Merit Score." From the website:

"TalentSpring creates accurate merit scores by using the votes from candidates. Advanced mathematics are used to detect inaccurate votes and remove them while still accurately ranking candidates. The top resume in an industry receives a merit score of 2,000 and the most entry level candidate receives a merit score of 1,000."

How does it work?

"The voting process used to generate the Merit Score rankings is very simple. Voters are shown a series of resume pairs. With each pair the voter is asked which of the two candidates is most likely to be brought into an interview for the typical job opening in this job category. It is that simple - is Candidate A or Candidate B better in this job category. There is no worrying about previous pairs or what resumes are going to show up next. Each pair is considered in isolation."

And who's voting?

"Your resume is voted on by other people seeking to be ranked in the same category you are. Just as you are voting on other candidates in the same job category you are in. Since TalentSpring offers quite a few job categories to choose from, on occasion you may be voting on (and be voted on by) candidates in related job categories. For example, a C++ programming candidate might end up voting on Java programmers."

What about accuracy?

"We know when people are voting outside the "normal" range and remove these votes from the ranking calculations. We think that the ability to accurately vote is a skill that recruiters are interested in because it reflects both your understanding of the position you are interested in and your attention to detail. That is why we calculate and post your voting score as part of your Candidate Overview."


So what do you think? I was disappointed at who's doing the ranking--I assumed by "peers" they meant one's actual co-workers. Now that would be interesting, given that these type of peer ratings are at least partially trustworthy. I wonder how accurate ratings of competing job hunters will be. With no control over subject matter expertise, this relies solely on (what I assume is) statistical abnormalities. Not particularly encouraging. In addition, selecting one person over another for a general job category may prove to be an impossible task, as even jobs in a single category can vary substantially in terms of competencies/KSAOs required.

BUT, that said, it is encouraging to see steps taken in a different direction. If we could just combine some of these approaches, we may be making our way slowly toward a database that employers could feel good about. Of course that means a whole other discussion on rating formats...

Hat tip.

Becoming passive employers

Let's take a moment and think about what job search could be.

Right now, job search is static. Someone searches for a job, and either a vacancy exists or it doesn't. But what if we were a little more creative?

What if instead of getting "zero results for your search", the candidate received something like:

There are no current openings that match your search. However, the following positions exist that may have openings in the future.

What followed would be a detailed description of current positions in the organization that matched the search criteria--jobs people actually had. And you would allow people to submit a job interest request so they would be notified when that job (or similar job) became open. Yes, some systems already have job interest requests, but too often it's based on broad job titles and it fails to provide the rich information a job seeker needs (e.g., who they will work with, learning opportunities).

What else could we do with this feature? We could profile the individuals that are in the current job. Okay, maybe not everyone, but a sample. At the very least we could provide a basic job description (and not a boring one).

This idea fits with a concept I think we all need to focus more on. In addition to seeking passive candidates, we should be passive employers. Passive job seekers aren't looking for a job, but they could be. Passive employers don't have that particular opening--but they could. But unless you tell candidates that, how will they know? How do they know that a perfect match exists in your organization, and if they just had waited another week to search, they would have seen it?

Why do we make applicants the servants of the ATS, not the other way around?

Let's take this a step further. Let's say I'm an attorney in Seattle looking to relocate to Boston. I know I'd like to work for a smallish firm with decent billable hours, co-workers that know their stuff and are good at their jobs but value work-life balance.

How the HECK am I supposed to find that firm? Sure, I can look for current vacancies on job boards. Or maybe I just happen to know someone who works for such a firm and they have an opening. Or I might be able to find some information through a Google search or services such as Vault or Hoover's (although that information is very limited, you still have to know the company name, and information on public sector agencies is anemic). But that'll only get me so far. Then what?

There is no general database of employer qualities to search through (sites like Jobfox are trying a similar idea but it's still based on vacancies). No easy way to punch in the above criteria and have a system spit out, "Here are all the firms that meet your criteria. Here are the ones that currently have openings, here are the ones that don't currently may may in the future."

People search is getting more and more sophisticated. What about employer search? If we expect applicants to take an active role in managing their career, we should give them the information they need to do it. We can, and should, do better.

Job ads of the future?

Curious about the direction job postings are going?

Looking for ways to snazz up your postings?

Then read this post over at jobs2web. Check out the graphic.

How close are your postings and/or career portals to this? Are they even in the same ballpark?

How hard would it be to add things like:

- links to a webinar/job preview video

- RSS feed

- subscribe to similar jobs

Answer: not hard. Let's hurry up and get there!

Protuo takes candidate matching to new level

A few posts back I talked about Jobfox (formerly Market10), a great website that does much better than most job boards at matching up applicants with job opportunities. This type of technology is definitely a dominant trend right now, and we're seeing many products out there that really try to narrow down the applicant funnel in a speedy but valid way (Trovix is another example). Another trend is coupling these 'smart funneling' systems with simulation tests, such as the Virtual Job Tryout or Virtual Job Audition.

In an article in this month's Inc. Magazine, another similar type of service is described--Protuo.

Protuo's job matching system is probably the most detailed I've seen, which is good or bad depending upon how you look at it. Good in that the matches that are generated are hypothetically much more likely to be on target. Bad because given most people's attention span, I question how many people will actually fill out the entire profile.

The candidate/job profile has three broad categories:

* Personal skills and knowledge
* Social skills and training
* Business and analytical skills

Within each category is another set of fillable tabs. For example, under "Personal skills and knowledge" there are separate categories for Personal Skills (e.g., creativity, initiative) and Arts, Language, and Science (e.g., math, biology). Individuals can create their own web pages to highlight their qualifications (example here)--with full HTML editing capability and blog included, which are nice touches.

Aside from time considerations, there's another potential problem with drilling so deep. People aren't known for their ability to describe themselves particularly accurately. This is not helped by the fact that people are asked to describe their skill level in general terms that are not tailored to the category and for which no instructions are provided (e.g., Basic, Intermediate, Advanced, Expert). My guess is the match percentage will suffer for it.

On the other hand, these types of systems are still developing. Credit goes to Protuo for combining job search with personal pages. Expect much more in the way of tailored application and assessment processes that try to maximize speed and quality.

Jobfox plays matchmaker

I've said before that I think the "holy grail" of selection will be matching quality measures of candidate values, interests, and competencies with those required for a position in a particular organization. (And I'm not alone here)

While job search search sites and applicant tracking software provide generic matching capability--say, letting candidates or employers search by full-time/part-time, geographic preference, etc.--I haven't seen anything more impressive, even though we know quite a bit about person-job and person-organization fit and its importance--until now.

Jobfox (previously Market10), developed by CareerBuilder founder and former CEO Rob McGovern, is taking things a step further by allowing job seekers and employers to find each other using a variety of "dimensions" (10, to be specific), including some of the aforementioned:

- Previous employment

- Education

- Skills (tied to particular employment)

- Desired salary

- Willingness to travel

Pretty basic stuff, right? Well here's where things get interesting. Look at the additional pieces of information a candidate can enter:

- Growth stage of company you want to work for (Start-up/Growing/Established)

- Dress code preference (Business Professional/Business Casual/Casual)

- Size of employer preferred (<500,>500)

- Location you work best from (home, work, part of each)

- Employer type desired (for profit, not for profit, government)

- Benefits desired (yes this seems basic but I don't recall seeing this before)

The other thing that's very different from traditional job boards is the job seeker doesn't SEARCH for jobs. Instead, results are generated based on data the individual enters/attaches and how well it matches specific opportunities.

From the employer's perspective, they get (for a fee) candidates whose profile best matches position/organization needs.

Less clutter to sift through for both sides--I love it. I see this as a great step toward eventually doing a much better of job of matching candidates and employers.

Results are looking positive with various employers signed up. Unfortunately right now they only serve the Atlanta and Washington, D.C. markets but they will be expanding to the San Francisco Bay Area in May, and Boston by June.

Downside? This is still active job seekers and employers. The next big challenge will be to not only have a database of valid individual assessment results (and accurate job descriptions) but to have this information for passive as well as active job seekers. And not just current job openings, but information for all employers. What would this look like? I'm not sure. Maybe an outgrowth of a popular social networking site, such as MySpace, mixed with organizational data from, say, Vault? All I know is it will be fun to watch!

Employer brochure available here. Other information, including press releases, here.

Podcasting and jobs

Podcasts are one of the technology tools de jour. Whether it will stick around for the long haul is up for debate, but for right now there are a number of ways they can be used effectively.

Advertise your opportunities

One way is to use podcasts as another way to get the word out about your organization and career opportunities. One firm that facilitates this is jobsinpods. Check out this example of how this content is already becoming popular in Google searches.

Gather job information

Another way to use them is to gather information about positions, say as part of a job analysis. Take a listen to this example, "Life as an Escalation Engineer" with Microsoft. I think you'll find it's a rich source of information about typical tasks and job requirements.

Can you think of other ways to podcast?