While reading news about a court decision recently I was struck again by how the U.S. Court system continues to make a false and largely unhelpful distinction between "objective" and "subjective" assessment processes, and they're certainly not the only ones. Presumably this is meant to highlight how some are based on judgment while others are free from it.
One last time: it's all subjective.
I challenge you to come up with a single aspect of the personnel assessment process that is not based on judgment. Not only that, "degree of judgment" is not a useful metric in determining the validity of a selection process--for many reasons including but not limited to whose judgment is being used.
Here is a sample of assessment components that are based on judgment:
- how to conceptualize the job(s) to be filled
- how to study the job(s)
- how to recruit for the job(s)
- which subject matter experts to use
- how to select the KSAs to measure
- how to measure those KSAs
- how items are created or products are selected
- how to set a pass point, if used
- how to administer the assessments
- how to score the assessments
- how to combine assessments
- how to make a final selection decision
- what type of feedback to give the candidates
And this is just the tip of the iceberg. The entire process is made up, like all complex decisions, of smaller decisions that can impact the selection process (what software do I use to analyze the data? Is this one KSA or two?).
So what WOULD be helpful in describing how an assessment process is developed and administered? I can think of a few yardsticks:
1. Extent to which processes and decisions are based on evidence. To what extent is the assessment and selection process based on methods that have been shown scientifically to have significant utility?
2. Degree of structure. To what extent are the assessment methods pre-determined? How flexible are the processes during the assessment?
3. Multi-dimensionality. How many KSAs are being measured? Is it sufficient to predict performance?
4. Measurement method. How many assessment methods are being used? Do they make sense given the targeted KSAs?
5. Transparency. Is the entire process understandable and documented?
I'm not inventing the wheel here. I'm not even reinventing it. I'm pointing out how most cars have four of them. It's obvious stuff. But it's amazing to me that some continue to perpetuate artificial distinctions that fail to point out the truly important differences between sound selection and junk.
3 comments:
Bryan, I usually find myself in agreement with your posts, but here I find myself in disagreement. I am not a lawyer, so I will not try to engage in a legal analysis. However, I think even a simple example will show the problem with your argument.
Consider a state where if I prescribe medicine I need to be a Physician and to be a Physician required an MD. Now, at a certain level, a subjective decision has been made by the state. Now we have a Medical Director in that State who decides to hire a Physician for a Hospital. The Medical Director decided to hire the first Physician that shows up with an MD. If the MD requirement is challenged, has the Physician made a subjective decision? Has the Hospital made a subjective decision? Does it assist the court to say - well all the decisions are more or less subjective?
Now consider a very common case involving some issues similar to Dukes v Walmart. The head of HR for a company that owns many independent stores. The company and the head of HR decides that all stores hire employees at minimum wage plus a cost of living increase every year; that is it for the compensation policy. After 10 years female employees sue alleging that there is disparate impact and sex discrimination as a result of this policy. Has everyone engaged in subjective decision making - the individual managers, the stores, the HR director, the company? If a Judge asked you as an expert would you say - yes, they all did because all decisions are more or less subjective? Does such a judgment assist the court, given that decisions over over class actions and discrimination theories may all rest on a finding regarding subjectivity?
Dennis - so glad you responded. I was hoping my rant would spark some discussion.
Primarily I'm responding to the widespread use of the word "subjective" as if it has some universally understood meaning and is clearly distinct from an "objective" process. It takes well-thought responses from people like you to point out that the situation is more complicated than I made it out to be.
Now to your points. In the case of the hiring of a physician, I don't think the use of an MD as a "screen" on the part of the Medical Director is subjective--in fact that wasn't even his/her decision. On the other hand, the decision to hire the first person to show up was, and that's what I would question (e.g., how was the position recruited for? was it word of mouth, which may favor certain groups who were therefore more likely to show up first?). As was the decision to hire a Physician (versus, say, a nurse practitioner).
As for Walmart, if the plaintiffs are challenging solely the minimum wage policy (I'm not as familiar with the case as you are), I would think the primary decision makers would be on the hook (e.g., head of HR). My understanding is they were also challenging the promotion practices.
But your point is well-taken. When a selection practice is challenged, where does the judgment about judgment lie?
Perhaps another layer could be focused on the extent of discretion an individual decision maker had--I see this as more neutral than the word "subjective" which somehow (to me at least) implies decisions made in the absence of evidence (otherwise, wouldn't they be objective?). Of course I am still much more interested in HOW and WHY they made those decisions, not the fact that they had the ability to make the decision in the first place.
Hi,
I read through the judgement. I seriously feel that case has hints of discrimination. Reason could be sex discrimination or even that they want to avoid a litigation happy employee. Subjectivity cannot be ruled out from any decision, but what we want to reduce is Bounded rationality as well as Arbitrariness.
Post a Comment