Many of the presentations from the 2013 Association of Test Publishers conference are now available and I strongly recommend checking them out. Strong themes include gamification, global assessment, item development, and test security, just to name a few.
I found the one from HumRRO on creating a virtual role-play fascinating. If you haven't already seen it, I strongly recommend you check out their demo at www.humrro.org/simdemo.html. The username and password are both humrrodemo. A small taste of what's possible in assessment!
Hat tip.
Saturday, February 23, 2013
Monday, February 18, 2013
Research update addendum
Okay, I know I just did a research update but but I had a couple stragglers, including a pretty important one, namely the Spring 2013 Personnel Psychology, which is free right now!
- First, an important piece by Bobko and Roth updating d values for Black-White differences on selection measures. Their updated analysis indicates measures such as biodata and assessment centers may have d values as large as paper-and-pencil tests of cognitive ability. Personality measures still benefit from small differences. They include a helpful table that breaks down the values by construct, and they also include a list of factors that can impact d, such as job complexity and range restriction.
- Second, a fascinating study of aberrant personality tendencies and their impact on career outcomes conceptualized using the Five-Factor Model and measured using the NEO PI-R. More evidence that "dark side" personality traits are an important consideration in predicting career trajectories.
The other that just came out is the February 2013 Journal of Applied Social Psychology:
- First, a look at moderators of the relationship between employee weight and job-related outcomes.
- Second, a study that I think has implications for selection: looking at circumstances under which competitors copy their opponents choices. I've observed over and over again that when an employee gets a competing offer, suddenly their attractiveness increases. Perhaps not the same phenomenon, but worth exploring.
- Next, another study of discrimination, this time age discrimination in within- and between-career job changes. Results indicated discrimination against older workers was particularly pronounced when older applicants were making between- rather than within-career changes.
- Okay, I'm sensing a theme to this post. In this study, the authors looked at how the wording of occupational descriptions activates gender stereotypes.
- Finally, something not about discrimination: the authors of this (small N) study found that a perceived aspect of emotional intelligence predicted perceived negotiation success beyond traditional personality traits.
And on that note...until next time!
- First, an important piece by Bobko and Roth updating d values for Black-White differences on selection measures. Their updated analysis indicates measures such as biodata and assessment centers may have d values as large as paper-and-pencil tests of cognitive ability. Personality measures still benefit from small differences. They include a helpful table that breaks down the values by construct, and they also include a list of factors that can impact d, such as job complexity and range restriction.
- Second, a fascinating study of aberrant personality tendencies and their impact on career outcomes conceptualized using the Five-Factor Model and measured using the NEO PI-R. More evidence that "dark side" personality traits are an important consideration in predicting career trajectories.
The other that just came out is the February 2013 Journal of Applied Social Psychology:
- First, a look at moderators of the relationship between employee weight and job-related outcomes.
- Second, a study that I think has implications for selection: looking at circumstances under which competitors copy their opponents choices. I've observed over and over again that when an employee gets a competing offer, suddenly their attractiveness increases. Perhaps not the same phenomenon, but worth exploring.
- Next, another study of discrimination, this time age discrimination in within- and between-career job changes. Results indicated discrimination against older workers was particularly pronounced when older applicants were making between- rather than within-career changes.
- Okay, I'm sensing a theme to this post. In this study, the authors looked at how the wording of occupational descriptions activates gender stereotypes.
- Finally, something not about discrimination: the authors of this (small N) study found that a perceived aspect of emotional intelligence predicted perceived negotiation success beyond traditional personality traits.
And on that note...until next time!
Saturday, February 09, 2013
Research update
It's that time, February research update--and there are some really fascinating studies out there:
First the March 2013 issue of IJSA:
- Procedural fairness is an important aspect of organizational attraction, especially for rejected applicants.
- Does personality type impact perceptions of the selection process? According to this study and their measures (Bohemian is a personality type?), sometimes.
- Do you use SEM to generate a SED and ultimately score bands? Then you will definitely want to read this piece, which suggests the traditional method is flawed.
- Emotional intelligence continues to be a hot topic. In this study the authors describe the development of a SJT-based measure.
- Staying on the topic of personality, this study adds to the debate over the utility of constructs vs. facets in predicting performance--this time with conscientiousness predicting police officer performance.
- P-O fit is often used to explain why people apply for certain jobs. In this study researchers found that values that applicants find attractive are particularly important (as opposed to neutral or aversive aspects).
- I've written about automated reference check systems before. In fact I've written about the company that the first author of this study is from (SkillSurvey). What is partially a demonstration of the value of their product also adds to our knowledge of reference checks by finding no demographic differences and the ability to predict involuntary (but not voluntary) turnover.
- How big of a deal is it when it is discovered that an applicant "embellished" on their resume? Depends on how much you already liked them.
- In this study of applicants in Iran, the authors found that the usefulness of a web-based application system was more important than how easy it was to use. So basically looks are important but not sufficient.
Next, the January issue of the Journal of Applied Psychology:
- Construct validity has always been a sticky wicket. The authors of this study take an interesting approach, using video-based testing to illustrate a method for supporting construct validity.
- This study explores the variance in assessment center ratings stemming from various sources, and the authors argue that existing literature has masked this variance.
-The authors of this study show how the application of a particular bootstrap method applied to meta-analyses increases accuracy.
- Speaking of meta-analysis, the authors of this study demonstrate that the differences between coefficient alpha and composite reliability in actual data sets is quite small.
On to the January issue of the Journal of Applied Social Psychology, which as of the time of this post was available for free!
- Similar to findings in other areas, the impact of negative publicity is stronger than positive publicity on organizational attraction, according to the authors of this study. The authors also found in these cases, organizations would do well to provide detailed recruiting ads to mitigate the impact.
- Age discrimination seems to be increasing in popularity as a research topic (which makes sense from a demographic perspective). In this study, the authors found raters discriminated against older and younger applicants, with the former being the least likely to be hired.
- Here's an interesting one: in this study, people of color perceived White women to be, among other things, ditsy, shallow, privileged, and appearance focused. Stereotypes know no color boundaries.
Let's take a look at the February 2013 issue of the Journal of Personality and Social Psychology:
- First, a fascinating look at how generalizable the five-factor model of personality is across cultures. Specifically, the authors looked at a largely illiterate, indigenous culture in Bolivia. They failed to find support for the widely used model, and suggest that the structure of personality may differ across societies.
- Second, a study of sex and gender differences that points out the importance of considering the construct of gender (as opposed to the binary concept of sex) for a variety of behavioral and psychological indicators, including big five personality.
Let's move now to the February 2013 issue of Psychological Science, where:
- In this study, the authors found that age differences in cognitive functioning within cohorts were often as large as those between cohorts, which calls into question the cohort explanation for age differences.
- In a study that I think has implications for personnel assessment, the authors found that raters of M.B.A. applications that had already given several high recommendations were less likely to do so later in the day. Another bias to watch out for!
Last but not least, there was an article recently in the New York Times about how important referrals are for some organizations and the increased importance this places on social networking sites. There was a reference to a study done in August of 2012 on source of hire that I hadn't seen before, which found that not only were referrals more likely to be hired, they had longer tenure--which is consistent with other research I've seen. Unfortunately, no word about performance differences.
That's all for now!
First the March 2013 issue of IJSA:
- Procedural fairness is an important aspect of organizational attraction, especially for rejected applicants.
- Does personality type impact perceptions of the selection process? According to this study and their measures (Bohemian is a personality type?), sometimes.
- Do you use SEM to generate a SED and ultimately score bands? Then you will definitely want to read this piece, which suggests the traditional method is flawed.
- Emotional intelligence continues to be a hot topic. In this study the authors describe the development of a SJT-based measure.
- Staying on the topic of personality, this study adds to the debate over the utility of constructs vs. facets in predicting performance--this time with conscientiousness predicting police officer performance.
- P-O fit is often used to explain why people apply for certain jobs. In this study researchers found that values that applicants find attractive are particularly important (as opposed to neutral or aversive aspects).
- I've written about automated reference check systems before. In fact I've written about the company that the first author of this study is from (SkillSurvey). What is partially a demonstration of the value of their product also adds to our knowledge of reference checks by finding no demographic differences and the ability to predict involuntary (but not voluntary) turnover.
- How big of a deal is it when it is discovered that an applicant "embellished" on their resume? Depends on how much you already liked them.
- In this study of applicants in Iran, the authors found that the usefulness of a web-based application system was more important than how easy it was to use. So basically looks are important but not sufficient.
Next, the January issue of the Journal of Applied Psychology:
- Construct validity has always been a sticky wicket. The authors of this study take an interesting approach, using video-based testing to illustrate a method for supporting construct validity.
- This study explores the variance in assessment center ratings stemming from various sources, and the authors argue that existing literature has masked this variance.
-The authors of this study show how the application of a particular bootstrap method applied to meta-analyses increases accuracy.
- Speaking of meta-analysis, the authors of this study demonstrate that the differences between coefficient alpha and composite reliability in actual data sets is quite small.
On to the January issue of the Journal of Applied Social Psychology, which as of the time of this post was available for free!
- Similar to findings in other areas, the impact of negative publicity is stronger than positive publicity on organizational attraction, according to the authors of this study. The authors also found in these cases, organizations would do well to provide detailed recruiting ads to mitigate the impact.
- Age discrimination seems to be increasing in popularity as a research topic (which makes sense from a demographic perspective). In this study, the authors found raters discriminated against older and younger applicants, with the former being the least likely to be hired.
- Here's an interesting one: in this study, people of color perceived White women to be, among other things, ditsy, shallow, privileged, and appearance focused. Stereotypes know no color boundaries.
Let's take a look at the February 2013 issue of the Journal of Personality and Social Psychology:
- First, a fascinating look at how generalizable the five-factor model of personality is across cultures. Specifically, the authors looked at a largely illiterate, indigenous culture in Bolivia. They failed to find support for the widely used model, and suggest that the structure of personality may differ across societies.
- Second, a study of sex and gender differences that points out the importance of considering the construct of gender (as opposed to the binary concept of sex) for a variety of behavioral and psychological indicators, including big five personality.
Let's move now to the February 2013 issue of Psychological Science, where:
- In this study, the authors found that age differences in cognitive functioning within cohorts were often as large as those between cohorts, which calls into question the cohort explanation for age differences.
- In a study that I think has implications for personnel assessment, the authors found that raters of M.B.A. applications that had already given several high recommendations were less likely to do so later in the day. Another bias to watch out for!
Last but not least, there was an article recently in the New York Times about how important referrals are for some organizations and the increased importance this places on social networking sites. There was a reference to a study done in August of 2012 on source of hire that I hadn't seen before, which found that not only were referrals more likely to be hired, they had longer tenure--which is consistent with other research I've seen. Unfortunately, no word about performance differences.
That's all for now!
Subscribe to:
Posts (Atom)