Several research updates this time, including fascinating studies of how computers can assist with interview training, and how a brief writing exercise can lower stereotype threat for women:
International Journal of Selection and Assessment, December, 2016:
- International support for the cultural intelligence scale
- Looking to improve applicant interview performance? Maybe a computer can help.
- This study found that time lag and g-loading are important factors impacting re-testing results
- Status-seeking seems to be an important individual difference when looking at self-presentation behaviors, including exaggeration and faking in job search
- Development and validation of a 360-degree measure of leadership personality
Personnel Psychology, Winter 2016:
- Do CEOs significantly impact firm performance? This study found evidence that they do.
- More evidence that the assumption that performance is normally distributed is questionable
- A more accurate correction for range restriction is presented, and an example analysis indicates the relationship between the Big 5 and job satisfaction may be greater than previously believed
Journal of Organizational Behavior, November, 2016 (which includes several articles devoted to the importance of theory in organizational sciences):
- Are self-focused or other-focused recruiting advertisements more effective? This study suggests it depends not only on cultural differences but individual regulatory focus
Journal of Applied Psychology, October, 2016:
- A suggestion for improving meta-analytic structural equation modeling
Journal of Applied Psychology, November, 2016:
- A fascinating study of how having women compose a brief written description of their personal values can help ameliorate stereotype threat in competitive environments
Showing posts with label leadership. Show all posts
Showing posts with label leadership. Show all posts
Tuesday, November 15, 2016
Thursday, March 05, 2015
Mega research update
I hope you like research, because there's a lot of it coming your way...and many are free as of this posting!
Without further ado...
Let's start with the Journal of Applied Psychology, January 2015 issue:
- We see a lot of research involving large candidate groups, but much less for individuals. In this meta-analysis of individual assessments, the authors found support for their usefulness, but it varied significantly across studies. Highest validity was found for managerial jobs and assessments that included a cognitive ability test.
- Being in the wrong job can be frustrating for both the employee and the employers. In this study, the authors show a relationship between poor vocational fit and counterproductive work behaviors (CWBs).
- Speaking of CWB, there may be more of them going on than you would think based on the assessment literature...
- And even more on CWB! These authors found support for both self- and acquaintance-reported personality ratings, specifically conscientiousness and agreeableness, in predicting "workplace deviance".
- Unfortunately, gender bias still exists in selection. In this meta-analysis, the authors found this to be particularly the case in male-dominated jobs. On a positive note, they do suggest ways of mitigating this: provide clear evidence of the competence of applicants, encourage careful decision making, and use experienced raters.
- The over-/under-prediction of cognitive ability tests debate for different ethnic groups continues. In this study, the authors find support of overprediction for African Americans, suggesting the tests are not predictively biased.
Next, the March issue of J.A.P.:
- More support for the predictive validity of emotional intelligence, but more importantly, how the concept overlaps with other constructs such as the Big 5 and self-efficacy.
- All situational judgment tests (SJTs) are not equal, and according to these authors in a large number of instances the context that is presumably important? Not so much.
- Speaking of SJTs, these researchers suggest that putting the "situational" back in SJTs--i.e., assessing how the situation is analyzed rather than the response options--is a useful method.
- A fascinating update of effect size benchmarks that can be used for a variety of purposes.
- Trying to predict safety-related behavior? This research suggests that personality traits, particularly agreeableness, can usefully predict this behavior.
Moving on to the March issue of IJSA (free right now!):
- Some guidelines on preparing norms for personality inventories.
- Evidence that different cultures have different procedural justice perceptions of different selection mechanisms
- Some important findings on the equivalence and stability of job performance ratings over time
- Development of a new measure of subjective career success
- More evidence that both technical knowledge and prosocial knowledge are important factors in predicting medical student clinical performance
- This study found that CWBs are under-reported and organizational commitment increases the likelihood that peers will report them
- Evidence that forced-choice and Likert-type scales used in personality inventories have similar measurement properties
On to the Spring issue of Personnel Psych (also free right now!):
- This meta-analysis on narcissism showed that it is related to leadership emergence (through extraversion) and leadership effectiveness in a curvilinear fashion.
- More evidence of the importance of political skill--particularly the aspects of networking ability, interpersonal influence, and apparent sincerity--in predicting a range of important outcomes, including task performance beyond GMA and the Big 5. It would be interesting to see how this is related to emotional intelligence (yes this is a foreshadowing).
Turning to the March issue of Psych Bulletin:
- More on narcissism: this time, researchers found that men consistently report higher levels of narcissism compared to women, which is interesting when taken in combination with the study above.
In the December issue of Industrial and Organizational Psychology:
- The first focal article calls out researchers for using incorrect assumptions about criterion reliabilities, thus impacting criterion validity values. They make suggestions for how to improve meta-analyses moving forward.
- The second makes the important argument that utility analyses should consider measures of well-being when determining the effectiveness of interventions (such as an employment test).
Finally, in the January issue of JOB (also free right now):
- a proposal for improving the calculation and reporting of Cronbach's alpha
- a fascinating study showing that high conscientiousness may hinder performance during stressful situations
- in support of EI, this study found a link between emotion recognition ability and income (interestingly through political skill and interpersonal facilitation...remember the earlier study on political skill?).
That's all for now!
Without further ado...
Let's start with the Journal of Applied Psychology, January 2015 issue:
- We see a lot of research involving large candidate groups, but much less for individuals. In this meta-analysis of individual assessments, the authors found support for their usefulness, but it varied significantly across studies. Highest validity was found for managerial jobs and assessments that included a cognitive ability test.
- Being in the wrong job can be frustrating for both the employee and the employers. In this study, the authors show a relationship between poor vocational fit and counterproductive work behaviors (CWBs).
- Speaking of CWB, there may be more of them going on than you would think based on the assessment literature...
- And even more on CWB! These authors found support for both self- and acquaintance-reported personality ratings, specifically conscientiousness and agreeableness, in predicting "workplace deviance".
- Unfortunately, gender bias still exists in selection. In this meta-analysis, the authors found this to be particularly the case in male-dominated jobs. On a positive note, they do suggest ways of mitigating this: provide clear evidence of the competence of applicants, encourage careful decision making, and use experienced raters.
- The over-/under-prediction of cognitive ability tests debate for different ethnic groups continues. In this study, the authors find support of overprediction for African Americans, suggesting the tests are not predictively biased.
Next, the March issue of J.A.P.:
- More support for the predictive validity of emotional intelligence, but more importantly, how the concept overlaps with other constructs such as the Big 5 and self-efficacy.
- All situational judgment tests (SJTs) are not equal, and according to these authors in a large number of instances the context that is presumably important? Not so much.
- Speaking of SJTs, these researchers suggest that putting the "situational" back in SJTs--i.e., assessing how the situation is analyzed rather than the response options--is a useful method.
- A fascinating update of effect size benchmarks that can be used for a variety of purposes.
- Trying to predict safety-related behavior? This research suggests that personality traits, particularly agreeableness, can usefully predict this behavior.
Moving on to the March issue of IJSA (free right now!):
- Some guidelines on preparing norms for personality inventories.
- Evidence that different cultures have different procedural justice perceptions of different selection mechanisms
- Some important findings on the equivalence and stability of job performance ratings over time
- Development of a new measure of subjective career success
- More evidence that both technical knowledge and prosocial knowledge are important factors in predicting medical student clinical performance
- This study found that CWBs are under-reported and organizational commitment increases the likelihood that peers will report them
- Evidence that forced-choice and Likert-type scales used in personality inventories have similar measurement properties
On to the Spring issue of Personnel Psych (also free right now!):
- This meta-analysis on narcissism showed that it is related to leadership emergence (through extraversion) and leadership effectiveness in a curvilinear fashion.
- More evidence of the importance of political skill--particularly the aspects of networking ability, interpersonal influence, and apparent sincerity--in predicting a range of important outcomes, including task performance beyond GMA and the Big 5. It would be interesting to see how this is related to emotional intelligence (yes this is a foreshadowing).
Turning to the March issue of Psych Bulletin:
- More on narcissism: this time, researchers found that men consistently report higher levels of narcissism compared to women, which is interesting when taken in combination with the study above.
In the December issue of Industrial and Organizational Psychology:
- The first focal article calls out researchers for using incorrect assumptions about criterion reliabilities, thus impacting criterion validity values. They make suggestions for how to improve meta-analyses moving forward.
- The second makes the important argument that utility analyses should consider measures of well-being when determining the effectiveness of interventions (such as an employment test).
Finally, in the January issue of JOB (also free right now):
- a proposal for improving the calculation and reporting of Cronbach's alpha
- a fascinating study showing that high conscientiousness may hinder performance during stressful situations
- in support of EI, this study found a link between emotion recognition ability and income (interestingly through political skill and interpersonal facilitation...remember the earlier study on political skill?).
That's all for now!
Sunday, January 04, 2015
2014 Research of the Year (+ research update)
Happy New Year! As I've done in previous years, I present below the research articles I ran across in this area that I think were the most impactful and/or important of 2014. But first, let's catch up on two issues:
First, the Winter issue of Personnel Psychology:
- Situational judgment tests have been shown to be useful for measuring interpersonal skills, but beware: levels of "angry hostility" moderate that relationship. (Is there a happy hostility?)
- When hiring leaders, should you look for those that have a busy home life, or be wary of them? In this fascinating study, the authors found that leader family-to-work conflict negatively impacts followers in that it can increase their burnout. However, family-to-work enrichment increased follower engagement through leader engagement. So the answer is, as usual, not simple: home/family life can be a good thing for followers if it makes the leader more engaged; but if the home/family life is increasing burnout, the leader may pass that along to others. So it would seem it all depends on how the individual is handling their life outside of work!
Let's look at the November issue of the Journal of Applied Psychology:
- Are men or women perceived as better leaders? According to this meta-analysis, it depends on how you ask the question. If you limit the question to other-ratings, women are rated significantly higher. But if you look at self-ratings, men rate themselves significantly higher. Which leads to the next question: is it a biological perception or a gender perception, and if the latter, what traits are the most important?
- An intriguing study of how applicant confidence interacts with and can be altered by the recruitment experience, in this case among recruits for the U.S. military.
- Next, a study of employment and job search efficacy. Not surprisingly, within-person frequency of job search behavior correlated with job offers; interestingly, the relationship between perceived job search progress and efficacy beliefs were moderated by beliefs of internal attribution.
- Last but not least, more evidence of the importance of defining the criteria when predicting job performance. In this meta-analysis, the researchers found more support for personality traits out-predicting cognitive ability in predicting counterproductive work behavior, that the two predictors are approximately equal in predicting organizational citizenship behaviors, and that cognitive ability outperforms personality when predicting task and overall performance. So do you want high task performance, OCBs, or do you want to avoid CWB? :) (of course the situation is even more complicated depending on whether you're looking at individual, team, leader performance, over what period of time, etc.)
Okay, on to the awards! Without further ado, here are my nominations for Research of the Year for 2014:
1) Important advancements in our understanding of weight-based discrimination at work: Vanhove & Gordon.
2) A study of applicants posting faux pas on their social networking sites: Roulin.
3) Two important looks at assessments delivered remotely via mobile devices: Arthur, Doverspike, Munoz, Taylor, & Carr, and Morelli, Mahan, & Illingworth.
4) Two fascinating looks at personality at work: Judge, Simon, Hurst, & Kelley; and Wille & De Fruyt
5) An excellent study of how effective staffing and training practices impact firm-level flexibility and adaptability: Kim & Ployhart.
6) An important study of the movement of impactful I/O researchers to business schools: Aguinis, Bradley, & Brodersen.
7) The relationship between conscientiousness and job performance is more accurately described as curvilinear: Carter, Dalal, Boyce, O'Connell, Kung & Delgado
Finally, honorable mention to two great developments in 2014: the change of some publishers to making access to articles more affordable, and the announcement of an additional journal, the Journal of Personnel Assessment and Decisions.
I'm continually amazed at the quality of thought and research in our area and the passion and practicality you exhibit. Here's to an amazing 2014 and more in 2015!
Friday, November 28, 2014
Why leadership in the public sector is harder to find--but more important
Occasionally I post about things that are related to recruitment and assessment, but not focused exclusively on them. This is one of those times.
I have the following quote from Valve Software's New Employee Handbook (a fascinating document) posted on my office door:
"Hiring well is the most important thing in the universe.
Nothing else comes close. It’s more important than breathing.
So when you’re working on hiring—participating in
an interview loop or innovating in the general area of
recruiting—everything else you could be doing is stupid
and should be ignored!"
The older I get, the more I wonder whether I should cross out "hiring" and write in "leadership". I just can't bring myself to do it because they're both so darn important.
But this post will be about leadership. Specifically, leadership in the public (i.e., government) sector. More specifically, lack of leadership and what to do about it. I don't pretend that leaders in the private sector are uniformly outstanding, but public sector is what I'm most familiar with.
First things first: in some important ways, leadership in the public sector (PS) is different from the private sector. Not night-and-day I grant you, but there are some relatively unique boundary conditions that apply, namely:
- Not only are PS leaders bound by normal organizational policies and procedures, they labor under an additional layer of laws and rules, whether federal, state, or local. Unlike policies and procedures, they cannot be easily changed--in fact in many cases this requires moving heaven and earth. As one (very important) example, typically there are laws/rules about how you can hire someone. Many of these laws and rules were created 50+ years ago in reaction to spoils systems and haven't been seriously evaluated since.
- Many PS employees have civil service protections. This isn't a bad thing, but it means moving employee that are bad fits (either over or out) is difficult. This greatly inhibits your talent mobility strategy.
- In the PS, leadership is often treated as an afterthought, rather than the linchpin upon which organizational success relies. The assumption seems to be that the organizational systems and processes are so strong that it almost doesn't matter who's in charge. This means things like leadership development and training are half-hearted.
These conditions combine with several other factors to result in true leadership being relatively rare in the PS:
- Failure is invisible. There is very little measurement of leadership success and very little transparency and accountability, absent a media storm.
- There are fewer people in management positions that have the important leadership competencies. Things like listening ability, strategic planning ability, and emotional intelligence. Instead they are usually chosen based on technical ability and without the benefit of rigorous assessment results.
- There is a lack of understanding of leadership. This stems from the lack of attention paid to it as a serious discipline; without operational definitions of leadership, there is no measurement and no accountability.
- An unwillingness to treat leadership seriously. For whatever reasons--politics, lack of motivation, entrenched cultures--leadership is relegated to second-class status when it comes to analyzing department/agency success. Focus tends to rest on line-level employees, technology, and unions--and only on top-level leaders when there is a phenomenally bad outcome.
So why is leadership more important in the PS than the private sector?
- Governments regulate many aspects of our lives. They're not making consumer products. Leaders in PS organizations have purview over things like public safety, the environment, education, housing, and taxes. Things you literally touch every day.
- There is less accountability, less transparency. PS leaders often do not report to a board. They don't have to produce annual reports that detail their successes and failures. What they do is often mysterious, poorly defined, and rarely sees the light of day.
- PS leaders work for you. Elected or not, their salary typically comes from taxpayers. They ultimately report to the citizens. This means you should care about what they're doing, and whether they are worthy stewards of your investment.
So what can be done? Much like the answer to "how do we hire well?", the answers are known. They're just not practiced very well:
1. Publicly acknowledge the scope of the problem. Like frogs in a pot, somehow we find ourselves in a situation where slowly over time the current situation is accepted as normal. It's time to stop pretending that all PS Managers are leaders. They're not. And we must look in the mirror ourselves and acknowledge that we are likely part of the problem.
2. Acknowledge the urgency to improve. Stop pretending that leadership is a secondary concern. Sub-par leadership has a negative impact on our lives every day. Improving the quality of that leadership is one of the most critical things we can do as a society.
3. Publicly commit to change, and actually follow through. Specifically describe what you will change, and when, and provide regular status updates.
4. Define leadership in measurable terms and behaviors. Here's just a sample list of what real leaders do (and not a particularly good one);
= continuously improve operations
= champion and reward innovation
= hold their people accountable for meeting SMART goals
= continually seek feedback and signs of their own success and failure
= create and sustain a culture that attracts high performers and dissuades poor fits
= make hiring and promoting the most qualified people THE most important part of their job
5. Hire and promote those with leadership competencies, not the best technicians. While knowledge of the work being performed is important, it is far from the most important competency.
6. Make the topic of leadership a core activity for every management team. Eliminate "information sharing" meetings and replace them with discussions on how to be better leaders.
7. Set clear goals of leaders up front, and hold them accountable. What does this mean?
= consequences for hiring poor fits
= consequences for poor morale on their team
= consequences for not setting and meeting SMART goals
= recognition for doing all of the above well
8. Measure leadership success and make the results transparent. Develop plans to address gaps and follow through.
9. Instill a culture of boldness and innovation. Banish fear, often borne of laboring under layers of red tape. Encourage risk-taking, and learn from mistakes rather than punishing them.
10. Relentlessly seek out and banish inefficiencies, especially related to the use of time. Critically evaluate how email and meetings are used; establish rules regarding their use.
11. Stop pretending that all of this applies only to first-line supervisors. If anything, they're more important the higher you go in the organizational chart.
12. When it comes to recruiting, stop focusing on low relative salaries, and capitalize on the enormous benefit of the PS an employer--namely the mission of public service.
13. View leadership as a competency, not a position. Leadership behaviors can be found everywhere in an organization--they should be recognized and promoted.
My intent here is not to be a downer, but to emphasize how much more focus needs to be placed on leadership in the public sector. The current state of affairs is unacceptable. And for those of us familiar with research and best practices in organizational behavior, it's painful.
So I apologize for the decidedly un-Thanksgivingy nature of this post. But I am thankful for free speech and open minds. Thanks for reading.
I have the following quote from Valve Software's New Employee Handbook (a fascinating document) posted on my office door:
"Hiring well is the most important thing in the universe.
Nothing else comes close. It’s more important than breathing.
So when you’re working on hiring—participating in
an interview loop or innovating in the general area of
recruiting—everything else you could be doing is stupid
and should be ignored!"
The older I get, the more I wonder whether I should cross out "hiring" and write in "leadership". I just can't bring myself to do it because they're both so darn important.
But this post will be about leadership. Specifically, leadership in the public (i.e., government) sector. More specifically, lack of leadership and what to do about it. I don't pretend that leaders in the private sector are uniformly outstanding, but public sector is what I'm most familiar with.
First things first: in some important ways, leadership in the public sector (PS) is different from the private sector. Not night-and-day I grant you, but there are some relatively unique boundary conditions that apply, namely:
- Not only are PS leaders bound by normal organizational policies and procedures, they labor under an additional layer of laws and rules, whether federal, state, or local. Unlike policies and procedures, they cannot be easily changed--in fact in many cases this requires moving heaven and earth. As one (very important) example, typically there are laws/rules about how you can hire someone. Many of these laws and rules were created 50+ years ago in reaction to spoils systems and haven't been seriously evaluated since.
- Many PS employees have civil service protections. This isn't a bad thing, but it means moving employee that are bad fits (either over or out) is difficult. This greatly inhibits your talent mobility strategy.
- In the PS, leadership is often treated as an afterthought, rather than the linchpin upon which organizational success relies. The assumption seems to be that the organizational systems and processes are so strong that it almost doesn't matter who's in charge. This means things like leadership development and training are half-hearted.
These conditions combine with several other factors to result in true leadership being relatively rare in the PS:
- Failure is invisible. There is very little measurement of leadership success and very little transparency and accountability, absent a media storm.
- There are fewer people in management positions that have the important leadership competencies. Things like listening ability, strategic planning ability, and emotional intelligence. Instead they are usually chosen based on technical ability and without the benefit of rigorous assessment results.
- There is a lack of understanding of leadership. This stems from the lack of attention paid to it as a serious discipline; without operational definitions of leadership, there is no measurement and no accountability.
- An unwillingness to treat leadership seriously. For whatever reasons--politics, lack of motivation, entrenched cultures--leadership is relegated to second-class status when it comes to analyzing department/agency success. Focus tends to rest on line-level employees, technology, and unions--and only on top-level leaders when there is a phenomenally bad outcome.
So why is leadership more important in the PS than the private sector?
- Governments regulate many aspects of our lives. They're not making consumer products. Leaders in PS organizations have purview over things like public safety, the environment, education, housing, and taxes. Things you literally touch every day.
- There is less accountability, less transparency. PS leaders often do not report to a board. They don't have to produce annual reports that detail their successes and failures. What they do is often mysterious, poorly defined, and rarely sees the light of day.
- PS leaders work for you. Elected or not, their salary typically comes from taxpayers. They ultimately report to the citizens. This means you should care about what they're doing, and whether they are worthy stewards of your investment.
So what can be done? Much like the answer to "how do we hire well?", the answers are known. They're just not practiced very well:
1. Publicly acknowledge the scope of the problem. Like frogs in a pot, somehow we find ourselves in a situation where slowly over time the current situation is accepted as normal. It's time to stop pretending that all PS Managers are leaders. They're not. And we must look in the mirror ourselves and acknowledge that we are likely part of the problem.
2. Acknowledge the urgency to improve. Stop pretending that leadership is a secondary concern. Sub-par leadership has a negative impact on our lives every day. Improving the quality of that leadership is one of the most critical things we can do as a society.
3. Publicly commit to change, and actually follow through. Specifically describe what you will change, and when, and provide regular status updates.
4. Define leadership in measurable terms and behaviors. Here's just a sample list of what real leaders do (and not a particularly good one);
= continuously improve operations
= champion and reward innovation
= hold their people accountable for meeting SMART goals
= continually seek feedback and signs of their own success and failure
= create and sustain a culture that attracts high performers and dissuades poor fits
= make hiring and promoting the most qualified people THE most important part of their job
5. Hire and promote those with leadership competencies, not the best technicians. While knowledge of the work being performed is important, it is far from the most important competency.
6. Make the topic of leadership a core activity for every management team. Eliminate "information sharing" meetings and replace them with discussions on how to be better leaders.
7. Set clear goals of leaders up front, and hold them accountable. What does this mean?
= consequences for hiring poor fits
= consequences for poor morale on their team
= consequences for not setting and meeting SMART goals
= recognition for doing all of the above well
8. Measure leadership success and make the results transparent. Develop plans to address gaps and follow through.
9. Instill a culture of boldness and innovation. Banish fear, often borne of laboring under layers of red tape. Encourage risk-taking, and learn from mistakes rather than punishing them.
10. Relentlessly seek out and banish inefficiencies, especially related to the use of time. Critically evaluate how email and meetings are used; establish rules regarding their use.
11. Stop pretending that all of this applies only to first-line supervisors. If anything, they're more important the higher you go in the organizational chart.
12. When it comes to recruiting, stop focusing on low relative salaries, and capitalize on the enormous benefit of the PS an employer--namely the mission of public service.
13. View leadership as a competency, not a position. Leadership behaviors can be found everywhere in an organization--they should be recognized and promoted.
My intent here is not to be a downer, but to emphasize how much more focus needs to be placed on leadership in the public sector. The current state of affairs is unacceptable. And for those of us familiar with research and best practices in organizational behavior, it's painful.
So I apologize for the decidedly un-Thanksgivingy nature of this post. But I am thankful for free speech and open minds. Thanks for reading.
Wednesday, August 06, 2014
Research update
I can't believe it's been three months since a research update. I was waiting until I got critical mass, and with the release of the September issues of IJSA, I think I've hit it.
So let's start there:
- Experimenting with using different rating scales on SJTs (with "best and worst" response format doing the best of the traditional scales)
- Aspects of a semi-structured interview added incremental validity over cognitive ability in predicting training performance
- Studying the use of preselection methods (e.g., work experience) prior to assessment centers in German companies
- The proposed general factor of personality may be useful in selection contexts (this one was a military setting)
- Evidence that effective leaders show creativity and political skill
- Investigating the relationship (using survey data) between personality facets and CWBs (with emotional stability playing a key role)
- Corrections for indirect range restriction boosted the upper end of structured interview validity substantially
- A method of increasing the precision of simulations that analyze group mean differences and adverse impact
- A very useful study that looked at the prediction of voluntary turnover as well as performance using biodata and other applicant information, including recruitment source, among a sample of call center applicants. Reuslts? Individuals who had previously applied, chose to submit additional information, were employed, or were referrals had significantly less voluntary turnover.
Moving on...let's check out the May issue of JAP; there are only two articles but both worth looking at:
- First, a fascinating study of the firm-level impact of effective staffing and training, suggesting that the former allow organizations greater flexibility and adaptability (e.g., to changing financial conditions).
- Second, another study of SJT response formats. The researchers found, using a very large sample, the "rate" format (e.g., "rate each of the following options in terms of effectiveness") to be superior in terms of validity, reliability, and group differences.
Next, the July issue of JOB, which is devoted to leadership:
- You might want to check out this overview/critique of the various leadership theories.
- This study suggests that newer models proposing morality as an important component of leadership success have methodological flaws.
- Last, a study of why Whites oppose affirmative action programs
Let's move to the September issue of Industrial and Organizational Psychology:
- The first focal article discusses the increasing movement of I/O psychology to business schools. The authors found evidence that this is due in large part to some of the most active and influential I/O researchers moving to business schools.
- The second is about stereotype threat--specifically its importance as a psychological construct and the paucity of applied research about it.
Coming into the home stretch, the Summer issue of Personnel Psych:
- The distribution of individual performance may not be normal if, as these researchers suggest, "star performers" have emerged
- Executives with high levels of conscientiousness and who display transformational leadership behavior may directly contribute to organizational performance
So let's start there:
- Experimenting with using different rating scales on SJTs (with "best and worst" response format doing the best of the traditional scales)
- Aspects of a semi-structured interview added incremental validity over cognitive ability in predicting training performance
- Studying the use of preselection methods (e.g., work experience) prior to assessment centers in German companies
- The proposed general factor of personality may be useful in selection contexts (this one was a military setting)
- Evidence that effective leaders show creativity and political skill
- Investigating the relationship (using survey data) between personality facets and CWBs (with emotional stability playing a key role)
- Corrections for indirect range restriction boosted the upper end of structured interview validity substantially
- A method of increasing the precision of simulations that analyze group mean differences and adverse impact
- A very useful study that looked at the prediction of voluntary turnover as well as performance using biodata and other applicant information, including recruitment source, among a sample of call center applicants. Reuslts? Individuals who had previously applied, chose to submit additional information, were employed, or were referrals had significantly less voluntary turnover.
Moving on...let's check out the May issue of JAP; there are only two articles but both worth looking at:
- First, a fascinating study of the firm-level impact of effective staffing and training, suggesting that the former allow organizations greater flexibility and adaptability (e.g., to changing financial conditions).
- Second, another study of SJT response formats. The researchers found, using a very large sample, the "rate" format (e.g., "rate each of the following options in terms of effectiveness") to be superior in terms of validity, reliability, and group differences.
Next, the July issue of JOB, which is devoted to leadership:
- You might want to check out this overview/critique of the various leadership theories.
- This study suggests that newer models proposing morality as an important component of leadership success have methodological flaws.
- Last, a study of why Whites oppose affirmative action programs
Let's move to the September issue of Industrial and Organizational Psychology:
- The first focal article discusses the increasing movement of I/O psychology to business schools. The authors found evidence that this is due in large part to some of the most active and influential I/O researchers moving to business schools.
- The second is about stereotype threat--specifically its importance as a psychological construct and the paucity of applied research about it.
Coming into the home stretch, the Summer issue of Personnel Psych:
- The distribution of individual performance may not be normal if, as these researchers suggest, "star performers" have emerged
- Executives with high levels of conscientiousness and who display transformational leadership behavior may directly contribute to organizational performance
Rounding out my review, check out a few recent articles from PARE:
- I'm not even gonna attempt to summarize this, so here's the title: Multiple-Group confirmatory factor analysis in R – A tutorial in measurement invariance with continuous and ordinal indicators
- Improving exploratory factor analysis for ordinal data
- Improving multidimensional adaptive testing
Last but not least, it's not related to recruitment or assessment, but check out this study that found productivity increases during bad weather :)
That's all folks!
- I'm not even gonna attempt to summarize this, so here's the title: Multiple-Group confirmatory factor analysis in R – A tutorial in measurement invariance with continuous and ordinal indicators
- Improving exploratory factor analysis for ordinal data
- Improving multidimensional adaptive testing
Last but not least, it's not related to recruitment or assessment, but check out this study that found productivity increases during bad weather :)
That's all folks!
Sunday, November 24, 2013
Research update
Well, it's that time of year again. No, not the holidays. No, not winter (or summer, depending on where you are!). Research update time! And I think you will agree with me that there is a lot of interesting research being reported, on traditional topics as well as emerging ones.
First, the November issue of JOB:
- Do transformational leaders increase creative performance and the display of OCBs? Well, that may depend on how much trait affectivity they had to begin with. A reminder to not make blanket statements like "X type of leadership causes Y type of behavior."
- There is seemingly endless debate about the utility of personality inventories. This study reminds us--again--that in assessment research there are few simple answers. The authors describe how a particular combination of personality measures correlated with task performance among professional employees, but not non-professionals. (yes, I said task performance)
Next, the Winter issue of Personnel Psychology (free right now!), much of which is devoted to corporate social responsibility (CSR):
- Do perceptions of CSR drive job pursuit intentions? It may depend on the applicant's previous justice experiences and their moral identity.
- Oh, and it may also depend on the extent to which applicants desire to have an impact through their work.
- There is a debate in the assessment center literature about whether competency dimensions are being measured or if it's purely a function of the assessment type. This study suggests that previous research has been hamstrung by a methodological artifact and that measured properly, assessment centers do in fact assess dimensions.
Let's switch to the November issue of the Journal of Applied Social Psychology:
- Engagement is all the rage, having seemingly displaced the age-old concept of job satisfaction (we'll see). This study reminds us that personality plays an important role in predicting engagement (so by extension our ability to increase engagement may be bounded).
- Here's another good one and it's related to internal motivations. The authors developed an instrument that helps organizations measure the "perception of the extant motivational climate." What does that mean? As I understand it, it's essentially whether most people are judging their performance against their peers or their own internal standards. It seems the latter may result in better results, such as less burnout.
- On to something more closely tied to assessment: letters of recommendation (LORs). There's surprisingly little research on these, but this study adds to our knowledge by suggesting that gender and racial bias can occur in their review, but requiring a more thorough review of them may reduce this (I don't know how likely this is for the average supervisor).
- Finally, a study looking at the evaluation of job applicants who voluntarily interrupted their college attendance. Unfortunately this does not appear to have been perceived as a good thing, and the researchers found a gender bias such that women with interrupted attendance had the lowest evaluations.
Next, the November issue of Industrial and Organizational Psychology, where the second focal article focus on eradicating employment discrimination. This article looks pretty juicy. I haven't received this one yet in the mail, so I may have more to say after digesting it. There are, as always, several commentaries following the focal article, on topics including background checks, childhood differences, and social networks.
Okay, let's tackle the 800-pound gorilla: the December issue of IJSA:
- Are true scores and construct scores the same? According to this Monte Carlo study, it seems how the scales were constructed makes a difference.
- Can non-native accents impact the evaluation of job applicants? Sure seems that way according to this study. But the effect was mediated by similarity, interpersonal attraction, and understandability.
- Here's a fascinating one. A study of applicants for border rangers in the Norwegian Armed Forces showed that psychological hardiness--particularly commitment--predicted completion of a rigorous physical activity above and beyond physical fitness, nutrition, and sensation seeking.
- Psst....recruiters...make sure when you're selling your organization you stay positive.
- Spatial ability. It's a classic KSA that's been studied for a long time, for various reasons including its tie to military assessments and the finding that measures can result in sex differences. But not so fast, spatial ability is not a unitary concept.
- Another study of assessment centers, this time in Russia and using a consensus scoring model.
- And let's round it out with one that should rock some worlds: the authors presents results that suggest that subject matter expert judgment of ability/competency importance bore little relation to test validity! Okay, I'm really curious about what the authors say about the implications, so if anyone reads this one, let us know!
Last but not least, the November issue of the Journal of Applied Psychology:
- Another on personality testing, this one underlining the important distinction between broad and narrow traits. This is another article I'm very curious about.
- Here's on one leadership: specifically, on the impact of different power distance values between leader and subordinates on team effectiveness
- And another on nonnative speakers! This one found discriminatory judgments made against nonnative speakers applying for middle management positions as well as venture funding. Interestingly, it appears to be fully mediated by perceptions of political skill--a topic that is hot right now.
- Okay, let's leave on a big note. This meta-analysis found an improvement in performance prediction of 50% when a mechanical combination of assessment data was used rather than a holistic (judgment-based) method. BOOM! Think about that the next time a hiring supervisor derides your spreadsheet.
Until next time!
First, the November issue of JOB:
- Do transformational leaders increase creative performance and the display of OCBs? Well, that may depend on how much trait affectivity they had to begin with. A reminder to not make blanket statements like "X type of leadership causes Y type of behavior."
- There is seemingly endless debate about the utility of personality inventories. This study reminds us--again--that in assessment research there are few simple answers. The authors describe how a particular combination of personality measures correlated with task performance among professional employees, but not non-professionals. (yes, I said task performance)
Next, the Winter issue of Personnel Psychology (free right now!), much of which is devoted to corporate social responsibility (CSR):
- Do perceptions of CSR drive job pursuit intentions? It may depend on the applicant's previous justice experiences and their moral identity.
- Oh, and it may also depend on the extent to which applicants desire to have an impact through their work.
- There is a debate in the assessment center literature about whether competency dimensions are being measured or if it's purely a function of the assessment type. This study suggests that previous research has been hamstrung by a methodological artifact and that measured properly, assessment centers do in fact assess dimensions.
Let's switch to the November issue of the Journal of Applied Social Psychology:
- Engagement is all the rage, having seemingly displaced the age-old concept of job satisfaction (we'll see). This study reminds us that personality plays an important role in predicting engagement (so by extension our ability to increase engagement may be bounded).
- Here's another good one and it's related to internal motivations. The authors developed an instrument that helps organizations measure the "perception of the extant motivational climate." What does that mean? As I understand it, it's essentially whether most people are judging their performance against their peers or their own internal standards. It seems the latter may result in better results, such as less burnout.
- On to something more closely tied to assessment: letters of recommendation (LORs). There's surprisingly little research on these, but this study adds to our knowledge by suggesting that gender and racial bias can occur in their review, but requiring a more thorough review of them may reduce this (I don't know how likely this is for the average supervisor).
- Finally, a study looking at the evaluation of job applicants who voluntarily interrupted their college attendance. Unfortunately this does not appear to have been perceived as a good thing, and the researchers found a gender bias such that women with interrupted attendance had the lowest evaluations.
Next, the November issue of Industrial and Organizational Psychology, where the second focal article focus on eradicating employment discrimination. This article looks pretty juicy. I haven't received this one yet in the mail, so I may have more to say after digesting it. There are, as always, several commentaries following the focal article, on topics including background checks, childhood differences, and social networks.
Okay, let's tackle the 800-pound gorilla: the December issue of IJSA:
- Are true scores and construct scores the same? According to this Monte Carlo study, it seems how the scales were constructed makes a difference.
- Can non-native accents impact the evaluation of job applicants? Sure seems that way according to this study. But the effect was mediated by similarity, interpersonal attraction, and understandability.
- Here's a fascinating one. A study of applicants for border rangers in the Norwegian Armed Forces showed that psychological hardiness--particularly commitment--predicted completion of a rigorous physical activity above and beyond physical fitness, nutrition, and sensation seeking.
- Psst....recruiters...make sure when you're selling your organization you stay positive.
- Spatial ability. It's a classic KSA that's been studied for a long time, for various reasons including its tie to military assessments and the finding that measures can result in sex differences. But not so fast, spatial ability is not a unitary concept.
- Another study of assessment centers, this time in Russia and using a consensus scoring model.
- And let's round it out with one that should rock some worlds: the authors presents results that suggest that subject matter expert judgment of ability/competency importance bore little relation to test validity! Okay, I'm really curious about what the authors say about the implications, so if anyone reads this one, let us know!
Last but not least, the November issue of the Journal of Applied Psychology:
- Another on personality testing, this one underlining the important distinction between broad and narrow traits. This is another article I'm very curious about.
- Here's on one leadership: specifically, on the impact of different power distance values between leader and subordinates on team effectiveness
- And another on nonnative speakers! This one found discriminatory judgments made against nonnative speakers applying for middle management positions as well as venture funding. Interestingly, it appears to be fully mediated by perceptions of political skill--a topic that is hot right now.
- Okay, let's leave on a big note. This meta-analysis found an improvement in performance prediction of 50% when a mechanical combination of assessment data was used rather than a holistic (judgment-based) method. BOOM! Think about that the next time a hiring supervisor derides your spreadsheet.
Until next time!
Sunday, May 19, 2013
Research update
Okay, I'm just gonna say it. There is an insane amount of research out there directly or indirectly related to our field. I'll be honest, it's a little daunting thinking about reading and processing all of it.
Luckily, you have me to overly simplify it for you so you can incorrectly describe it to others. So let's jump right in!
First up, the June issue of IJSA:
- Fairness perceptions matter in web-based selection too!
- The predictive validity of conscientiousness is moderated by self-enhancement
- Speaking of conscientiousness, it's a better predictor of performance in routinized jobs than those with complex cognitive requirements
- Psychological hardiness predicts adaptability in military leaders. This study is awesome also for showing a negative relationship between SAT scores and West Point performance.
- Recruiters, listen up: this study found that contingent workers converted to full-time status performed as well as referral and online hires, but not as well as those sourced internally (bookmark this one, there's a dearth of recruitment research)
- Scoring biodata: empirical keying reigns supreme over rational and quasi-rational approaches. Quasi-rational? Sounds like me most of the time.
- Personality and job performance: interactions are important beyond main effects
Okay, next, the May issue of JAP:
- Adaptive decision making in military leaders: both the brain and the mind are important (okay you monists out there, chill out)
- Are you a MANOVA (wo)man? Then read this.
- Proactive leaders set more challenging goals and have higher sales performance--assuming they have the trust of their subordinates.
Next, the April issue of JOB. Just one study, albeit interesting, in which female evaluators were less likely to recommend hiring or promoting Asian (versus White) applicants into jobs requiring social skills
Speaking of JOB, how about the May issue?
- Have you been calling for more research on calling? Your wish is granted.
- Curious about the concept of curiosity? (okay, I'll stop) Looks like it can be predictive of job performance above and beyond traditional cognitive and non-cognitive predictors. I'm gonna assume it varies with job, but I think the authors are probably right that it will increase in importance over time.
- When selecting for teams that may experience crisis situations (e.g., nuclear power plant crews), consider homogeneity--not mean levels--of positive affect.
- Moderate levels of supervisory structure combined with high levels of consideration lead to lowest levels of CWB's
- The dark side of OCBs
- The light side of OCBs
- Dark side and light side working together in a picture that has nothing to do with selection
Okay, onwards and upwards: one from the April issue of JASP, on generalized self-efficacy, work-related self-efficacy, and job-related outcomes
And one from the May issue: Sensation seeking and need for structure predict military field exercise performance
How about a little EI research? Haven't had a lot of that lately. Here's a piece from the May issue of Journal of Management that found emotion management ability to be a valid predictor of job performance.
Let's look in the May Psychological Science:
- Where we find a fascinating study that demonstrates assessment of profound cognitive abilities at a young age predicts outstanding contributions in many adults
- Another interesting study of how an individual responds to daily stressors (which I would posit are differentially found in various jobs) has long-term consequences for their mental health
- High schoolers with high math and verbal skills are less likely to choose STEM careers than those with math skills but moderate verbal skills
- Okay, check this one out: mindfulness training improved GRE reading comp scores as well as working memory. Implications for personnel assessment??
Shifting to the May Psychological Bulletin:
- Rorschach, anyone?
- Believing that individuals are malleable, rather than fixed ("implicit theory") predicts goal achievement. Implications for leaders?
Last but not least, let's hook up with our friends at PARE and see what they're up to:
- Into Excel? Check out this article on forest plots
- Or maybe factor analysis is your thing.
That's all folks! Hope your brain is spinning like mine!
Luckily, you have me to overly simplify it for you so you can incorrectly describe it to others. So let's jump right in!
First up, the June issue of IJSA:
- Fairness perceptions matter in web-based selection too!
- The predictive validity of conscientiousness is moderated by self-enhancement
- Speaking of conscientiousness, it's a better predictor of performance in routinized jobs than those with complex cognitive requirements
- Psychological hardiness predicts adaptability in military leaders. This study is awesome also for showing a negative relationship between SAT scores and West Point performance.
- Recruiters, listen up: this study found that contingent workers converted to full-time status performed as well as referral and online hires, but not as well as those sourced internally (bookmark this one, there's a dearth of recruitment research)
- Scoring biodata: empirical keying reigns supreme over rational and quasi-rational approaches. Quasi-rational? Sounds like me most of the time.
- Personality and job performance: interactions are important beyond main effects
Okay, next, the May issue of JAP:
- Adaptive decision making in military leaders: both the brain and the mind are important (okay you monists out there, chill out)
- Are you a MANOVA (wo)man? Then read this.
- Proactive leaders set more challenging goals and have higher sales performance--assuming they have the trust of their subordinates.
Next, the April issue of JOB. Just one study, albeit interesting, in which female evaluators were less likely to recommend hiring or promoting Asian (versus White) applicants into jobs requiring social skills
Speaking of JOB, how about the May issue?
- Have you been calling for more research on calling? Your wish is granted.
- Curious about the concept of curiosity? (okay, I'll stop) Looks like it can be predictive of job performance above and beyond traditional cognitive and non-cognitive predictors. I'm gonna assume it varies with job, but I think the authors are probably right that it will increase in importance over time.
- When selecting for teams that may experience crisis situations (e.g., nuclear power plant crews), consider homogeneity--not mean levels--of positive affect.
- Moderate levels of supervisory structure combined with high levels of consideration lead to lowest levels of CWB's
- The dark side of OCBs
- The light side of OCBs
- Dark side and light side working together in a picture that has nothing to do with selection
Okay, onwards and upwards: one from the April issue of JASP, on generalized self-efficacy, work-related self-efficacy, and job-related outcomes
And one from the May issue: Sensation seeking and need for structure predict military field exercise performance
How about a little EI research? Haven't had a lot of that lately. Here's a piece from the May issue of Journal of Management that found emotion management ability to be a valid predictor of job performance.
Let's look in the May Psychological Science:
- Where we find a fascinating study that demonstrates assessment of profound cognitive abilities at a young age predicts outstanding contributions in many adults
- Another interesting study of how an individual responds to daily stressors (which I would posit are differentially found in various jobs) has long-term consequences for their mental health
- High schoolers with high math and verbal skills are less likely to choose STEM careers than those with math skills but moderate verbal skills
- Okay, check this one out: mindfulness training improved GRE reading comp scores as well as working memory. Implications for personnel assessment??
Shifting to the May Psychological Bulletin:
- Rorschach, anyone?
- Believing that individuals are malleable, rather than fixed ("implicit theory") predicts goal achievement. Implications for leaders?
Last but not least, let's hook up with our friends at PARE and see what they're up to:
- Into Excel? Check out this article on forest plots
- Or maybe factor analysis is your thing.
That's all folks! Hope your brain is spinning like mine!
Saturday, December 01, 2012
Research update
Cold/wet yet? Well sit back, have some hot tea, and let's catch up on our research...
Let's start with the biggie: the December issue of IJSA.
- Juggling selection quality and adverse impact continues to be tricky. The authors in this article suggest an optimum combination.
- Reflecting results from the turnover literature, this study found a relationship between perceptions of promotion practices and organizational justice and job satisfaction.
- Speaking of justice, the authors of this study found that the relationship between perceptions of distributive justice and intentions to recommend an employer were moderated by applicant affect.
- Job seekers would to well to keep up their psychological well being and self-esteem (easier said than done, right?).
- Back to fairness. It's reasonable to think that ethnic minority applicants may not perceive video resumes well (due to the increased salience of their minority status). But at least in this study, that assumption was not supported--although it depended on ethnic identity and language proficiency.
- Honestly, I'm not one for the pure concept of "multi-tasking": in my experience people perform in serial, not parallel. But that doesn't stop people from researching the predictive validity of multi-tasking assessments.
- Faking of personality inventories has been one of the hottest topics in assessment for years, but is often framed as dichotomies. This study looks to bring some needed complexity to the issue using qualitative data.
- We all know it can be challenging to get hiring manager to give up their unstructured interviews of questionable validity. Interestingly, according to this study, the more a hiring manager has used unstructured interviews, the less open they are to change. I don't know if this is a causality issue, a mediating variable issue, or more evidence of the inability to accurately judge one's abilities.
- Need more evidence for discrimination that occurs during resume screening? Here ya go.
- Understanding why certain individuals perform better during interviews is a needed area for study. In this article, the authors demonstrate the importance of simply being ability to generate ideas, rather than analyzing the situational requirements.
- This study investigates another area needing more attention: the selection into medical training and education programs. The authors found strong predictive support for SJTs but they came with lower face validity.
- Last in this issue is another interesting study, this time of 360-ratings of innovation. Interestingly and unlike a lot of other research on self-perception, self-ratings were lower than overall observer ratings. However, the situation got more complex when the authors separated and analyzed by level of self-rating.
Next the November issue of Journal of Applied Social Psychology:
- One of the most pervasive (yet bizarrely under-discussed) areas of discrimination in the workplace is age discrimination. This study illustrates some of the stereotypes held of younger and older workers--by both groups. (Spoiler alert: you'll find out how accurate some of these are in just a second)
- I've been waiting for this one, partly because I love hearing about how bizarre and non-face valid some puzzle-based interviews are. This particular study was looking at perceptions of these interviews compared to a behavioral interview. Results? The puzzle-based interviews were consistently less popular. Oh, did I mention that they didn't work as well?
How about the Winter issue of Personnel Psych?
- Why do certain applicants withdraw from the recruitment process? This study suggests a relationship with organizational identification.
- Okay, back to stereotypes about older workers. Whereas the earlier study looked at what the stereotypes are, this one looks at whether they're true. The answer: no, but for one: older workers are less willing to participate in training and career development.
- Turns out it's not just the unemployed that are frustrated by the job search process--currently employed individuals feel the same in many ways. Boy it's too bad we don't have a giant shared database that is able to match job demands with worker abilities...wait...
The November issue of Journal of Applied Psychology has a couple gems:
- Evidence for the predictive validity of the external manifestations of personality as well as the associated implicit motives.
- A reminder that what makes for effective leadership behavior depends on the culture.
The latest issue of Personnel Review has an interesting research article on utility analysis, where the authors reiterate how challenging it can be to communicate UA information (hint: carrot and stick approach may work best).
There are a couple good ones in the November issue of Psychological Science.
- Multiple-choice tests have been beat up in the past for being nothing more than tests of recognition (rather than productive retrieval). This study presents evidence that refutes that assumption. Go multiple-choice!
- Conspiracy theorists take note: governments may be less likely to use the assumed relationship between genetic testing and intelligence to pigeon-hole us into tracks. Why? Because, at least according to this study, there appears to be little evidence connecting the two.
Okay, this one is pretty cool--in a slightly scary way. The authors were looking at the impact that virtual avatar attractiveness has on interview ratings. Turns out our bias toward attractive people is so strong it extends to the virtual world! Of course maybe I should have seen that coming...I mean, ever read a comic book? (hat tip)
Still with me? Last but not least, some disturbing new evidence regarding significance testing and potential publication bias (hat tip). I'm guessing most of you won't be surprised at the finding.
I don't know if I'll have another update before the end of the year, so if I don't, happy holidays to everyone!
Sunday, September 30, 2012
Research update: September, 2012
It's been a while since I've posted a research update. I was waiting for a critical mass but honestly it was taking a while! But no more waiting, let's see what's out there...
First up, a topic near and dear to my heart: HR certification. In the September/October issue of HRM, Lyons, et al. took a look at a sample of web-based HR job ads to study the prevalence of requiring the PHR/SPHR certifications offered by SHRM, er, the the Human Resource Certification Institute. Back in 2005, a similar study found that less than 2 percent of jobs listed the certifications as preferred or required. This time? 15.6 percent. Boom! Quite an increase (although obviously still far from the majority). Now the important question: does possession of said certificates actually predict job performance? I gotta be totally honest, I've seen some blind admiration of these certifications without any indication that this question was addressed.
Anyhoo, what else is in that issue of HRM? For one, a study of perceived supervisor support and team-level performance. Subjects: 75 gas stations in Norway (I just wanted to say that). Results? Link between the two. Implication? Another competency to consider when hiring supervisors. Oh, and btw, there's another study in the same issue about HR practices in the organization and links to OCB and customer service (hint: participation is good).
Like soccer (or, should I say, football)? How about team research? Either way you'll be interested in this fascinating study, which found that players transferred to another team improved the performance of their new team against their old one.
Finally, this small study which found having applicants complete self-affirming written self-guidance statements prior to interviewing improved their performance.
Okay, next up is the September issue of JAP.
How about another leadership study? Why not. In this one, researchers found supervisor consideration behaviors were positively related to employee attitudes (and apparently the more the merrier) while there was an ideal level of initiating structure behavior. There's a lot more with this one, particularly on P-E fit, so check it out.
Have you ever found yourself wondering, "I wonder what the power of cross-level interaction effects is when conducting tests of multilevel contingency and interactionism"? Well today's your lucky day, because check this out.
Promoting and fostering diversity in organizations is hypothesized to have various positive outcomes, one of which is creativity. But previous results are mixed. In this lab study, the authors propose an explanation: it depends on the extent to which team members take each others' perspectives. Another reminder that simply recruiting and hiring people with different backgrounds does not ensure successful performance.
Anyone performing differential item functioning (DIF) analysis should check out this study, which recommends a particular procedure for identifying optimal anchor items.
Finally, and this doesn't directly relate to assessment but is just darn interesting, a study on "cyberloafing" and the impact that moving to daylight savings time has on it (hint: it's not good).
Okay, back to the topic at hand and the October JOB.
Need another example of the complexity of the relationship between personality factors and organizational behavior? Check this out (personality factor: conscientiousness).
Speaking of conscientiousness, this study found that adaptive performance (acquiring new competencies as a result of organizational change) was related to task performance but the relationship was impacted by employee conscientiousness (and organizational politics!).
Did someone say conscientiousness? No, seriously, another study on it. But it's another interesting one that looks at the importance of group perceptions as well as team composition when analyzing the relationship between individual personality measures and performance. And it's another sports team study, this time university football players (U.S. football that is).
Last but not least, the October issue of JPSP. And the first study may just be the most interesting thing I've seen in a while.
In it, the authors demonstrate via lab and field experiments with a variety of subjects and decisions (including graduate school admission), that people seem to have a preference for potential rather than achievement. What are the implications for recruiting and assessment? That decision makers may be unduly influenced (at the cost of validity) by the promise of an applicant rather than their demonstrated accomplishments. However, this warning must be moderated with an acknowledgement that sometimes potential (e.g., ability scores) can be equally valid sources of predictive validity. So, bottom line: another bias to watch out for.
Next, an elucidating if slightly depressing study of stereotype threat. The authors demonstrate that the experience of stereotype threat among African Americans and Hispanic/Latino(a)s resulted in scientific disidentification and intention to pursue a scientific career.
Okay, this next one is tricky because it looks interesting but I wasn't able to find an in-press version (maybe a helpful reader can point one out?). It's a refinement of a theory of basic individual values, and looks like it has implications for career and applicant selection.
Finally, very last but most definitely not least, a study of the accuracy of personality judgments, a particularly timely topic given recent research that suggests these judgments have significant value in personnel assessment. Specifically, the authors were looking at hindsight effects (essentially how your perception changes after time and additional information). Good stuff and implications for anyone wanting to use observer measures of personality.
First up, a topic near and dear to my heart: HR certification. In the September/October issue of HRM, Lyons, et al. took a look at a sample of web-based HR job ads to study the prevalence of requiring the PHR/SPHR certifications offered by SHRM, er, the the Human Resource Certification Institute. Back in 2005, a similar study found that less than 2 percent of jobs listed the certifications as preferred or required. This time? 15.6 percent. Boom! Quite an increase (although obviously still far from the majority). Now the important question: does possession of said certificates actually predict job performance? I gotta be totally honest, I've seen some blind admiration of these certifications without any indication that this question was addressed.
Anyhoo, what else is in that issue of HRM? For one, a study of perceived supervisor support and team-level performance. Subjects: 75 gas stations in Norway (I just wanted to say that). Results? Link between the two. Implication? Another competency to consider when hiring supervisors. Oh, and btw, there's another study in the same issue about HR practices in the organization and links to OCB and customer service (hint: participation is good).
Like soccer (or, should I say, football)? How about team research? Either way you'll be interested in this fascinating study, which found that players transferred to another team improved the performance of their new team against their old one.
Finally, this small study which found having applicants complete self-affirming written self-guidance statements prior to interviewing improved their performance.
Okay, next up is the September issue of JAP.
How about another leadership study? Why not. In this one, researchers found supervisor consideration behaviors were positively related to employee attitudes (and apparently the more the merrier) while there was an ideal level of initiating structure behavior. There's a lot more with this one, particularly on P-E fit, so check it out.
Have you ever found yourself wondering, "I wonder what the power of cross-level interaction effects is when conducting tests of multilevel contingency and interactionism"? Well today's your lucky day, because check this out.
Promoting and fostering diversity in organizations is hypothesized to have various positive outcomes, one of which is creativity. But previous results are mixed. In this lab study, the authors propose an explanation: it depends on the extent to which team members take each others' perspectives. Another reminder that simply recruiting and hiring people with different backgrounds does not ensure successful performance.
Anyone performing differential item functioning (DIF) analysis should check out this study, which recommends a particular procedure for identifying optimal anchor items.
Finally, and this doesn't directly relate to assessment but is just darn interesting, a study on "cyberloafing" and the impact that moving to daylight savings time has on it (hint: it's not good).
Okay, back to the topic at hand and the October JOB.
Need another example of the complexity of the relationship between personality factors and organizational behavior? Check this out (personality factor: conscientiousness).
Speaking of conscientiousness, this study found that adaptive performance (acquiring new competencies as a result of organizational change) was related to task performance but the relationship was impacted by employee conscientiousness (and organizational politics!).
Did someone say conscientiousness? No, seriously, another study on it. But it's another interesting one that looks at the importance of group perceptions as well as team composition when analyzing the relationship between individual personality measures and performance. And it's another sports team study, this time university football players (U.S. football that is).
Last but not least, the October issue of JPSP. And the first study may just be the most interesting thing I've seen in a while.
In it, the authors demonstrate via lab and field experiments with a variety of subjects and decisions (including graduate school admission), that people seem to have a preference for potential rather than achievement. What are the implications for recruiting and assessment? That decision makers may be unduly influenced (at the cost of validity) by the promise of an applicant rather than their demonstrated accomplishments. However, this warning must be moderated with an acknowledgement that sometimes potential (e.g., ability scores) can be equally valid sources of predictive validity. So, bottom line: another bias to watch out for.
Next, an elucidating if slightly depressing study of stereotype threat. The authors demonstrate that the experience of stereotype threat among African Americans and Hispanic/Latino(a)s resulted in scientific disidentification and intention to pursue a scientific career.
Okay, this next one is tricky because it looks interesting but I wasn't able to find an in-press version (maybe a helpful reader can point one out?). It's a refinement of a theory of basic individual values, and looks like it has implications for career and applicant selection.
Finally, very last but most definitely not least, a study of the accuracy of personality judgments, a particularly timely topic given recent research that suggests these judgments have significant value in personnel assessment. Specifically, the authors were looking at hindsight effects (essentially how your perception changes after time and additional information). Good stuff and implications for anyone wanting to use observer measures of personality.
Saturday, May 26, 2012
May mega research update
It's time once again for the monthly research round-up. So let's dive right in:
The June International Journal of Selection and Assessment doesn't disappoint; let's take a look:
- More evidence of the link between personality variables and CWBs; this time with concurrent data in China
- Dovetailing nicely with a post I've been working on regarding promotional testing, this research indicates some interesting characteristics of internal test takers
- Why are supervisors open to behavioral interviews but shun discussion of "structure"? Looks like how we communicate about them plays a big role.
- More research on self-efficacy, this time teasing apart the concept a bit.
- Always a popular topic: applicant reactions to selection mechanisms. This time with a sample from Saudi Arabia.
- Speaking of applicant reactions...how about another study? This one comparing U.S. and Vietnamese college students. By the way, not surprisingly work samples came out a winner in both of these studies.
- Next, a fascinating study of a hidden bonus to UIT: despite the cheating element, it likely increases your candidate pool and eventually performance outcomes
- Speaking of response distortion, here's another study, this time of military cadet selection using personality inventories
- Okay, one more on inflation. This time a study of Chinese applicants--no difference compared to American samples.
- Back in March I wrote about a study Jeremy Bernerth published in J.A.P. that got a lot of attention. This time, Bernerth studied ethnic differences and found minority status was negatively related to credit scores.
Moving on to the summer issue of Personnel Psychology:
- The "file drawer problem" is the theory that nonsignificant results are less likely to get published. According to this study, that appears unlikely. But IMHO looking at all correlations is different than looking at the correlations key to one's hypothesis(es)...
- Back to faking (that may be this post's theme!), can response elaboration reduce faking on biodata items? This study suggests so. Although I'm left wondering...what was the impact on validity?
- Speaking of biodata, there are various ways of keying these items. This research suggests the best method depends on your sample size, although rational keying performed the worst.
How about the May issue of Journal of Applied Psychology?
- Well this is interesting...Chad Van Iddekinge and his colleagues have provided an updated meta-analysis on the criterion-related validity of integrity tests. What did they find? Well, the results appear to be less promising than those published previously (e.g., corrected r=.18 for job performance). Much like SIOP's research journal, this time J.A.P. published several commentaries in response to the study that...well, let's just say a debate ensued about the analysis...
- The Dark Triad. It sounds like something in a Dan Brown novel. But in this meta-analysis the authors show that personality characteristics that make up this triad (Machiavellianism, narcissism, and psychopathy) explain some variance in CWBs.
- Why are some people more proactive in seeking career goals than others? It's an important and under-researched question. In this study the authors show that part of the explanation lies in "future work selves", or how people's hopes and aspirations as they relate to work.
- Think self-reports of CWBs are biased? Perhaps not, according to this new study.
- Interested in what causes proactive customer service behavior? According to this multi-national study, self-efficacy is a key (along with service climate).
- Why do some leaders engage in more self-interested behavior than others? Perhaps not surprisingly, it appears due in part to the strength of their moral identity.
The May issue of the Journal of Applied Social Psychology has a couple gems...
- Hey, look, turns out being sensitive to your subordinates pays off. Talk about a lesson that needs frequent repeating...
- And that's it. Oh, wait, just this little study about using Facebook profiles to predict job performance...that I wrote about before....available in FULL right now...
Okay, getting to the end...The May/June issue of HRM:
- An interesting study of adverse impact in promotion decisions for managers in a Fortune 500 retailer. The authors compared three methods (top-down assessment, assessment centers, and multisource appraisal) and the results demonstrate how complex these analyses are!
- Speaking of complex. Think that successful job postings on the web is just fancy graphics? Think again--it still involves some classic factors like the labor market, firm reputation, and compensation incentives. The more things change...
- Identifying future leaders. There are few other issues that are as important for most organizations. Yet how exactly to do it eludes many. These authors propose a model that focuses on four main features: analytical ability, learning agility, drive, and emergent leadership.
Finally, a few from PARE:
- Does item order impact response anxiety? Not according to this study.
- What's that? How do we use a new jacknife procedure to eliminating items and improve structural equation modeling? You're in luck.
- Looks like a lot of research rely on beta weights when interpreting and reporting multiple linear regression results. But there's so much more...
Happy reading!
The June International Journal of Selection and Assessment doesn't disappoint; let's take a look:
- More evidence of the link between personality variables and CWBs; this time with concurrent data in China
- Dovetailing nicely with a post I've been working on regarding promotional testing, this research indicates some interesting characteristics of internal test takers
- Why are supervisors open to behavioral interviews but shun discussion of "structure"? Looks like how we communicate about them plays a big role.
- More research on self-efficacy, this time teasing apart the concept a bit.
- Always a popular topic: applicant reactions to selection mechanisms. This time with a sample from Saudi Arabia.
- Speaking of applicant reactions...how about another study? This one comparing U.S. and Vietnamese college students. By the way, not surprisingly work samples came out a winner in both of these studies.
- Next, a fascinating study of a hidden bonus to UIT: despite the cheating element, it likely increases your candidate pool and eventually performance outcomes
- Speaking of response distortion, here's another study, this time of military cadet selection using personality inventories
- Okay, one more on inflation. This time a study of Chinese applicants--no difference compared to American samples.
- Back in March I wrote about a study Jeremy Bernerth published in J.A.P. that got a lot of attention. This time, Bernerth studied ethnic differences and found minority status was negatively related to credit scores.
Moving on to the summer issue of Personnel Psychology:
- The "file drawer problem" is the theory that nonsignificant results are less likely to get published. According to this study, that appears unlikely. But IMHO looking at all correlations is different than looking at the correlations key to one's hypothesis(es)...
- Back to faking (that may be this post's theme!), can response elaboration reduce faking on biodata items? This study suggests so. Although I'm left wondering...what was the impact on validity?
- Speaking of biodata, there are various ways of keying these items. This research suggests the best method depends on your sample size, although rational keying performed the worst.
How about the May issue of Journal of Applied Psychology?
- Well this is interesting...Chad Van Iddekinge and his colleagues have provided an updated meta-analysis on the criterion-related validity of integrity tests. What did they find? Well, the results appear to be less promising than those published previously (e.g., corrected r=.18 for job performance). Much like SIOP's research journal, this time J.A.P. published several commentaries in response to the study that...well, let's just say a debate ensued about the analysis...
- The Dark Triad. It sounds like something in a Dan Brown novel. But in this meta-analysis the authors show that personality characteristics that make up this triad (Machiavellianism, narcissism, and psychopathy) explain some variance in CWBs.
- Why are some people more proactive in seeking career goals than others? It's an important and under-researched question. In this study the authors show that part of the explanation lies in "future work selves", or how people's hopes and aspirations as they relate to work.
- Think self-reports of CWBs are biased? Perhaps not, according to this new study.
- Interested in what causes proactive customer service behavior? According to this multi-national study, self-efficacy is a key (along with service climate).
- Why do some leaders engage in more self-interested behavior than others? Perhaps not surprisingly, it appears due in part to the strength of their moral identity.
The May issue of the Journal of Applied Social Psychology has a couple gems...
- Hey, look, turns out being sensitive to your subordinates pays off. Talk about a lesson that needs frequent repeating...
- And that's it. Oh, wait, just this little study about using Facebook profiles to predict job performance...that I wrote about before....available in FULL right now...
Okay, getting to the end...The May/June issue of HRM:
- An interesting study of adverse impact in promotion decisions for managers in a Fortune 500 retailer. The authors compared three methods (top-down assessment, assessment centers, and multisource appraisal) and the results demonstrate how complex these analyses are!
- Speaking of complex. Think that successful job postings on the web is just fancy graphics? Think again--it still involves some classic factors like the labor market, firm reputation, and compensation incentives. The more things change...
- Identifying future leaders. There are few other issues that are as important for most organizations. Yet how exactly to do it eludes many. These authors propose a model that focuses on four main features: analytical ability, learning agility, drive, and emergent leadership.
Finally, a few from PARE:
- Does item order impact response anxiety? Not according to this study.
- What's that? How do we use a new jacknife procedure to eliminating items and improve structural equation modeling? You're in luck.
- Looks like a lot of research rely on beta weights when interpreting and reporting multiple linear regression results. But there's so much more...
Happy reading!
Saturday, January 28, 2012
January 2012 research update
Okay, we've got a lot of ground to cover so let's jump right in...
- For you stats fans: Estimating ordinal reliability for Likert-type and ordinal response data
- And more: Effect size: Don't just report it, describe it
- And still more: A confidence interval approach to testing comparability of groups
- Shifting gears: CWBs: There's more than one type
- Femininity and military leadership
- Increasing HR's strategic participation
- HR practices do have an influence on organizational attractiveness
- IQ scores may be influenced by test motivation (in low-stakes settings)
- Learning agility is an important factor in identifying high-potentials
- Slicing and dicing the Big Five
- Extraversion, intelligence, and genetics
- Apparently being agreeable won't make you rich (especially if you're a man)
- Need scientists or engineers? Consider innovativeness and internal LOC
- What if you're looking for leaders of scientists or engineers? Consider three important criteria.
- Increase diversity cues to increase recruitment website stickiness
- Does transformational leadership relate to proactive behavior? Depends on the situation.
- Core self evaluations: More complicated than you might think
- No, seriously, it's much more complicated and involves general versus specific measures
- Speaking of complicated, the relationship between personality traits (Big 5) and performance isn't as simple as many make it out to be
That's all for now! See you in February.
- For you stats fans: Estimating ordinal reliability for Likert-type and ordinal response data
- And more: Effect size: Don't just report it, describe it
- And still more: A confidence interval approach to testing comparability of groups
- Shifting gears: CWBs: There's more than one type
- Femininity and military leadership
- Increasing HR's strategic participation
- HR practices do have an influence on organizational attractiveness
- IQ scores may be influenced by test motivation (in low-stakes settings)
- Learning agility is an important factor in identifying high-potentials
- Slicing and dicing the Big Five
- Extraversion, intelligence, and genetics
- Apparently being agreeable won't make you rich (especially if you're a man)
- Need scientists or engineers? Consider innovativeness and internal LOC
- What if you're looking for leaders of scientists or engineers? Consider three important criteria.
- Increase diversity cues to increase recruitment website stickiness
- Does transformational leadership relate to proactive behavior? Depends on the situation.
- Core self evaluations: More complicated than you might think
- No, seriously, it's much more complicated and involves general versus specific measures
- Speaking of complicated, the relationship between personality traits (Big 5) and performance isn't as simple as many make it out to be
That's all for now! See you in February.
Tuesday, December 27, 2011
Final research update of 2011
Welcome to the last HR Tests research update of 2011! This year has been filled with research developments, building on previous thinking as well as venturing out into new areas. Let's see what the end of the year brings us (hint: a lot)...
We'll start with the Winter 2011 issue of Personnel Psychology:
- If you're looking for executives who excel at strategic thinking, you'll want to pay attention to not only their cognitive ability but their accumulated work history, according to Dragoni, et al.
- I tend to think of realistic job previews (RJPs) as occurring pre-hire, but research by Earnest, et al. suggests that an effective technique is to conduct an oral and written RJP post-hire--and be as honest as possible.
- Meta-analyses are relied upon heavily as summaries of large bodies of research. Roth, et al. point out ways we can make them better, particularly with respect to estimates of validity and group differences.
- One of the downsides of cognitive ability tests is they're not always perceived very well by applicants. Sumanth & Cable show how this perception is influenced by the status of the organization as well as individual status.
- Next, a fascinating description of a scale to measure the idea of calling by Dobrow and Tosti-Kharas. What I found most interesting was how the measure was associated with different criteria for different domains (e.g., art, music, management). You can read a draft version here.
Let's turn now to the January issue of JOB:
- Political skill is a hot topic, and Gentry, et al. demonstrate how the perception of promotability related to this skill varies between bosses, peers, and direct reports. Long story short: it differs, and also depends on behavior (attitude will only take you so far, right?). You can read the full version (at least right now!) here.
- Anyone that's worked with (or has in their family) engineers or scientists knows that they often share some strong traits. This suggests leading these groups that engage in creative activity may require specific attributes, which is what Robledo, et al. set out to describe.
On to the January issue of JOM:
- Core self-evaluation (CSE) is another hot topic, and Chang, et al. provide an important review of 15 years of research, including the meta-analytic support for CSE predicting in-role and extra-role performance.
- Boswell, et al. provide an integrative review of the concept of job search across different situations (e.g., following job loss, while employed).
- One area that deserves more attention is selection into the highest positions within organizations. Withers, et al. provide a review of the process of selecting a director for a board.
Let's not forget v41(2) of Personnel Review:
- Hoglund delves into the topic of talent management as a strategic HRM practice. A fascinating topic that reinforces the importance of HRM as an influence over employee perceptions and behavior.
- The wording of job ads can have important impacts on applicant perceptions and behavior. De Cooman and Pepermans analyze the differences between for-profit and non-profit job ads, and show how only a fraction of the information potentially relevant for job-person match is published.
- Another topic that deserves additional attention is the motivation to expatriate. Altman and Baruch describe results of a qualitative study that may be useful to organizations thinking about attracting and selecting for positions that require this substantial move.
Don't forget this one:
- Johnson, et al. with an important reminder that when looking at the issue of discrimination, using single categories to define groups is probably not the best strategy.
Whew! And last but certainly not least, in the December issue of IOP, Michael McDaniel and colleagues present an argument (similar to ones made elsewhere) that the Uniform Guidelines are outdated and, worse than that, a detriment to the field of selection. The commentaries are many and range from support to passionate disagreement, with a healthy dose of caution (and dare I say...intransigence?) thrown in. Worth a read, particularly for those following the Guidelines literally and those engaged in related litigation. You can read a draft version here.
I hope everyone has a great New Years; here's to a wonderful 2012!
Saturday, December 10, 2011
Forget leadership assessment centers: Use the fat face test
All that time your organization spends on leadership assessment? Throw it out. New research indicates that CEOs with fat faces perform better, so just get out your measuring tape.
Okay, I'm oversimplifying. But this research by Wong and her colleagues is fascinating.
They looked at 55 photos of male CEOs and linked facial width to firm performance. Results? CEOs with higher "facial ratios" (face width relative to face height) led organizations that had "significantly greater firm financial performance."
Why facial width? Previous research has indicated this characteristic is related to aggression and sense of power. People who feel powerful tend to focus more on the big picture rather than little details, and tend to be better at staying on task.
Why men? The relationship between face ratio and behavior has been found to be important only in men. Something to do with testosterone.
But wait, don't throw out your work sample tests quite yet. The researchers also found that the relationship between facial ratio and firm performance was moderated by the style of the leadership team. The relationship was stronger in teams that were "cognitively simple" and saw things in black-or-white terms, apparently due to deference to authority. Not exactly something most organizations dream of. Presumably leadership teams with more nuanced views of the world rely on things beyond what shape their leader's head is.
Yes, I'm being tongue-in-cheek here. But if nothing else, this type of research reinforces the complex and important relationships between genetics, perceptions, and behavior.
Sunday, June 12, 2011
Mega research update
Time for a big research update. There's a lot to catch up on, so I'll go quickly and give you just enough for you to follow up on if you're interested.
- More support for the attraction-selection-attrition model
- A 25-year review of the study of leadership outcomes
- Faces with stereotype-relevant features are more open to prejudice
- Early socioeconomic experience plays big role in future risk-taking behavior
- Taking the perspective of another may be key to reducing racial bias
- Generational differences in Big 5 personality factors
- How to improve management research
- Pros and cons of using social networking sites to make HR decisions
- Research directions for talent management
- Organizational branding and "best employer" surveys
- Using social networking sites for hiring decisions (if we just had a single database...)
- How computer adaptive testing aids in delivering unproctored internet tests
- How broad dimension factors may improve assessment centers
- Practical intelligence predicts success as an entrepreneur
- Psychological capital (efficacy, hope, optimism, resilience) is related to job performance
- Expressions of personality factors varies with the situation
Last but not least, this gem, a meta-analysis of the efficacy of simulation games for instruction. Relevance for us? As active engagement went up, so did learning. Lessons? Consider interactive simulations for recruitment and selection, but make sure the viewer is truly involved and not just a spectator.
- More support for the attraction-selection-attrition model
- A 25-year review of the study of leadership outcomes
- Faces with stereotype-relevant features are more open to prejudice
- Early socioeconomic experience plays big role in future risk-taking behavior
- Taking the perspective of another may be key to reducing racial bias
- Generational differences in Big 5 personality factors
- How to improve management research
- Pros and cons of using social networking sites to make HR decisions
- Research directions for talent management
- Organizational branding and "best employer" surveys
- Using social networking sites for hiring decisions (if we just had a single database...)
- How computer adaptive testing aids in delivering unproctored internet tests
- How broad dimension factors may improve assessment centers
- Practical intelligence predicts success as an entrepreneur
- Psychological capital (efficacy, hope, optimism, resilience) is related to job performance
- Expressions of personality factors varies with the situation
Last but not least, this gem, a meta-analysis of the efficacy of simulation games for instruction. Relevance for us? As active engagement went up, so did learning. Lessons? Consider interactive simulations for recruitment and selection, but make sure the viewer is truly involved and not just a spectator.
Sunday, February 27, 2011
Spring '11 P-Psych: Leadership, interviews, competency modeling
It's journal season--this time let's take a look at the Spring 2011 issue of Personnel Psychology:
First up, Derue, et al. with an important meta-analysis on leadership effectiveness. After looking at 59 studies, they found that leader traits and behaviors explained a minimum of 31% of leadership effectiveness. Interestingly, group performance (which I would argue is the most important criterion) was the most difficult to predict. Behaviors tended to explain more than traits, but the authors suggest a model where behavior mediates the relationship between traits and effectiveness is warranted. Not surprisingly, the best trait predictor depended on the criterion: conscientiousness predicted leader effectiveness, group performance, and follower job satisfaction the best, while satisfaction with leader was best predicted by leader agreeableness (reminds me of a recent IJSA study). The same was true with leader behaviors, although consideration was a good predictor across the criteria. A must for anyone interested in leadership research, and you can read an in-press version here.
Next, a piece by Melchers, et al. on whether more interview structure really leads to better rating quality (I'll ruin it for you: yes). Specifically, using a sample of primarily undergraduates, the authors found that providing subjects with frame-of-reference (FOR) training and descriptively anchored rating scales led to substantial increases in rater accuracy and interrater reliability. You can read an in-press version here.
Those of you interested in the concept of core self-evaluations will want to read the study by Ferris, et al.
One area that we don't see enough research in is newcomer adaptation. The longitudinal study by Wang, et al. of a group of Chinese subjects helps fill that hole by exploring the relationship between adaptability, person-environment fit, and work-related outcomes.
Last but most certainly not least, Campion, et al. provide us with a review of best practices in competency modeling. Specifically, 20 of them. Of particular note to you may be that they distinguish competency modeling from job analysis. Want to read the whole thing? Me too! Wish I could have found an in-press version but no such luck.
Sidenote: while not specifically related to recruitment or selection, Christian, et al.'s piece on engagement may be of interest to several readers.
First up, Derue, et al. with an important meta-analysis on leadership effectiveness. After looking at 59 studies, they found that leader traits and behaviors explained a minimum of 31% of leadership effectiveness. Interestingly, group performance (which I would argue is the most important criterion) was the most difficult to predict. Behaviors tended to explain more than traits, but the authors suggest a model where behavior mediates the relationship between traits and effectiveness is warranted. Not surprisingly, the best trait predictor depended on the criterion: conscientiousness predicted leader effectiveness, group performance, and follower job satisfaction the best, while satisfaction with leader was best predicted by leader agreeableness (reminds me of a recent IJSA study). The same was true with leader behaviors, although consideration was a good predictor across the criteria. A must for anyone interested in leadership research, and you can read an in-press version here.
Next, a piece by Melchers, et al. on whether more interview structure really leads to better rating quality (I'll ruin it for you: yes). Specifically, using a sample of primarily undergraduates, the authors found that providing subjects with frame-of-reference (FOR) training and descriptively anchored rating scales led to substantial increases in rater accuracy and interrater reliability. You can read an in-press version here.
Those of you interested in the concept of core self-evaluations will want to read the study by Ferris, et al.
One area that we don't see enough research in is newcomer adaptation. The longitudinal study by Wang, et al. of a group of Chinese subjects helps fill that hole by exploring the relationship between adaptability, person-environment fit, and work-related outcomes.
Last but most certainly not least, Campion, et al. provide us with a review of best practices in competency modeling. Specifically, 20 of them. Of particular note to you may be that they distinguish competency modeling from job analysis. Want to read the whole thing? Me too! Wish I could have found an in-press version but no such luck.
Sidenote: while not specifically related to recruitment or selection, Christian, et al.'s piece on engagement may be of interest to several readers.
Monday, January 10, 2011
Why aren't more supervisors held accountable for their hiring mistakes?
Before I begin this rant...err...post, let's create a fantasy world.
No, not the kind with elves and magic (maybe in a future post).
The kind where organizations run efficiently, consistently, and rationally (see I told you it was going to be a fantasy).
So we'll set it up with two assumptions:
1) Job performance is defined for every individual in the organization in clear, meaningful ways.
2) All employees are held accountable for their critical decisions.
Okay, have this fantasy world in your mind? Great. Now that we have that out of the way, I ask you: Why aren't more supervisors in this world held accountable for their hiring decisions?
Think about it. What would you do with a supervisor who repeatedly:
- Made bad decisions about what technology to purchase/implement
- Made ineffective choices regarding how to divvy up team assignments
- Spent time inefficiently and chose to focus on low-priority projects
- Conducted meetings that were too long and had little payoff
Presumably write them up or get rid of them. All of these decisions have to do with the use of resources. Yet I could make a strong argument that the most important resource decision supervisors make is about who to bring into the organization.
So honestly, why aren't more supervisors held accountable? I have several theories:
1) The hires are blamed, not the supervisor. In some cases this makes sense. But when a hiring supervisor demonstrates a pattern of getting it wrong, someone needs to be looking at the decision-makers.
2) The connection between the decision and the consequences is distant. It's easier when a decision has immediate consequences, but when the employee has been around for 2 months, it's a lot easier to focus on their mistakes rather than how they got there to begin with.
3) It's harder to hold supervisors accountable. I think most organizations simply have a harder time quantifying or otherwise seeing supervisory performance in objective ways. Even when performance is measured, too often it focuses on how they manage their employees rather than their selection decisions.
4) People have a hard time giving negative feedback, especially to supervisors. It's hard enough to tell someone they're not meeting specific production goals; it's even harder to pinpoint exactly what the supervisor is doing wrong when hiring people--and then communicate that.
5) It's political. Organizations are typically even more committed to supervisors than rank-and-file because the decision of who to put there likely involved folks higher up in the organization. Thus, to admit a mistake in the supervisor is to admit a mistake in who hired them. Not to mention that supervisors are part of the very management structure that has issues in the first place.
6) It's not clearly measured as part of the job of a supervisor. Okay, I realize this is somewhat against the rules of our fantasy world, but I suspect this is often a big problem. Supervisors are usually measured against things like "leadership" or "administrative ability", not "the percentage of new hires who receive excellent performance ratings after 6 months."
So what can we do about this? Well I think some of the answers present themselves when we see what some of the problems might be above. Clearly document that hiring success will be considered a key performance metric for supervisors. Provide them training on the topic and feedback. Have consequences for those who keep getting it wrong (which may simply be taking the decision largely out of their hands).
Should HR be more involved? Almost certainly. It is after all HR's duty to maximize human performance in an organization. We're supposed to be the experts on this stuff--if we can't articulate the issue and suggest alternatives, why would we expect line management to?
Finally, let me end on something else to think about: Does your organization praise supervisors who repeatedly make good hiring decisions? We know from basic psychology that positive reinforcement is more likely to result in repeat behavior than punishment or negative reinforcement. But which do we normally use?
Now back to reality...
Subscribe to:
Posts (Atom)