Judgment and Human Perception

Judgment aims to find "true value," but what this means can differ for each person. Our minds act like measuring tools, using judgments to assess things. A judgment is a final decision, not a debate. On one side of the scale, there's computation, while on the other, there's taste and opinion. In between lies the domain of judgment.

Making a good judgment isn't the same as always having good judgment. Judgments don't cover matters of taste, which are personal and diverse. Judgment seeks true value, but what's considered valuable varies from one individual to another. The variability in human judgment makes us prone to mistakes.

There are two types of judgments, each posing its challenges when inconsistency arises:

1. Predictive Judgments: These involve making forecasts based on probabilities. For instance, if two doctors or weather forecasters reach starkly different conclusions using the same data, it suggests a problem known as "noise."
2. Evaluative Judgments: These judgments are based on personal values and preferences. Noise in this context means decisions that seem arbitrary and don't align with agreed-upon criteria.

Bias and Noise in Decision-Making

Imagine a group of shooters aiming at a target. Biased shooters consistently predictably miss the bull’s-eye, like always shooting to the left. Noisy shooters, however, scatter their shots randomly, making it hard to tell if they're even aiming at the target.

Bias means consistently getting results that predictably are off the mark, like a scale that always says you weigh five pounds more than you do. Noise, on the other hand, means results that vary randomly around an average, such as a manager who sometimes underestimates and sometimes overestimates how long a project will take.

Noise creeps in when there's conflicting information to interpret. For instance, two people might have different opinions about a problem even if they know the same things. They have to guess at the answer, weighing possibilities and assigning probabilities because there's no single right solution. Take a job candidate, for example, who might be a bit difficult but also really smart and ambitious. How can you predict if they'll make a good CEO? One study found that predictions about people's success ranged from 10% to 95% accurate.

The Reliability of Mechanical vs. Clinical Judgment

When we compare mechanical judgment with clinical judgment, a clear winner emerges in terms of reliability. Mechanical judgment excels because it eliminates the complexity and randomness inherent in human decision-making.

Predictive judgments, like those made in hiring processes, are crucial. They give insight into the "noise" present in decision-making. Professionals tend to make more errors compared to machines or simple rules. To measure this error, we use a "percent concordant" in a noise audit. This metric helps us compare clinical and mechanical judgments to see which is more accurate.

For instance, consider assessing job eligibility for two candidates. Mechanical judgment, despite its constraints and equal weighting of factors, remains reliable. Human judgment often relies on intuitive factors that can make decision-making seem random. While you might believe human judgment is more nuanced, factors like mood, timing, and personal preferences can't match the precision of mechanical predictions.

The Power and Peril of AI

In today's world, artificial intelligence (AI) has taken the spotlight for its ability to forecast outcomes using massive amounts of data. AI outperforms humans in accuracy when it comes to predicting random events. While humans often forgive their own mistakes, they demand near-perfection from machines. Unfortunately, human reliance on intuition often leads to avoidable errors.

Prediction and ignorance go hand in hand, more than we realize. Acknowledging what we don't know is the first step in dealing with uncertainty. It's better to admit ignorance than to let overconfidence lead to mistakes and misinformation.

Decision-Making Noise

When people quickly come to conclusions, they tend to stick with them. This happens because they might simplify a tough question, let their biases influence their judgment, or form quick impressions and refuse to reconsider. These biases create what's called system noise, which is the variation caused by everyone having different biases.

When you're faced with hard decisions, your brain aims to make judgments that seem right and better than any alternatives. But sometimes what you believe, and what you think others believe, don't match up – especially if your mood affects your thinking. These discrepancies contribute to what's known as pattern noise. It's made up of two types: stable pattern noise, which sticks around, and occasion noise, which comes and goes.

Errors and Noise in Judgments

When making judgments, three things can mess things up: how much weight you give to different factors, your reactions, and just being unique. Oh, and don't forget your own experiences and quirks—they can throw a spanner in the works too, making your judgments even more unpredictable. But hey, at least they'll probably be consistent with your personality.

Let's talk about mistakes. They come in three types:

1. Bias and system noise.
2. Level noise and pattern noise, are part of system noise.
3. Within pattern noise, there's stable pattern noise and occasion noise.

Noise is the real troublemaker here. It's more of a problem than bias. Among all the noise types, pattern noise takes the cake. It's way more common than level noise, usually showing up twice as often.

Enhance Judgment Quality with “Decision Observers”

Improving judgments can be done by using "decision observers" to minimize bias. One effective way is to have multiple judges evaluate the same problems to identify variations in their judgments, which we call noise. If there's too much noise in the system, consider using straightforward rules or algorithms instead of relying solely on individuals. Remember, though, that AI can't completely replace human judgment.

When selecting judges to improve accuracy, it's essential to start with those known for their good judgment. Experienced judges with a track record of making sound decisions tend to be confident in their assessments and can clearly explain their reasoning over the years. Look for judges who are thoughtful in their approach, questioning information to ensure its accuracy and credibility. They are also more receptive to feedback and willing to change their minds based on new facts. During a noise audit, these individuals can play a crucial role in identifying potential biases in decision-making processes.

Improve  “Decision hygiene”: Preventing Noise Before It Happens

When it comes to making decisions, preventing mistakes is way better than fixing them later. Just like washing hands to keep germs away, there are ways to keep "noise" out of our decisions. Noise is like random errors that sneak in and mess things up. Here's how we can practice "decision hygiene" to stop noise before it messes things up:

1. Giving Info in the Right Order: Imagine you're solving a mystery. If you get all the clues at once, you might jump to the wrong conclusion. But if you get them step by step, you're less likely to mess up. So, give out info only when needed and make sure everyone writes down their thoughts as they go along.

2. Taking Averaged Opinions: Think of predicting the weather. Sometimes one weatherperson is wrong, but if you ask a bunch of them and average their guesses, you're more likely to get it right. It's the same with decisions. Instead of relying on just one person's opinion, gather a few and average them out. This helps to reduce mistakes.

3. Using Diagnostic Guidelines: Doctors use guidelines to diagnose illnesses. It's like following a recipe. When everyone follows the same steps, it's harder to mess up. So, having clear guidelines for making decisions helps to cut down on errors.

Improve  Decision-Making for Employee Performance and Hiring

Performance reviews can be a headache.  They've become super complicated over time, but are they helpful? Not really. They don't truly tell us how good someone is at their job. But wait, there's a trick to make them better! Instead of using those confusing scales, why not just rank employees against each other? This cuts down on all the confusion and gives us more accurate results.

When it comes to hiring, there's another problem: interviewers. They bring all their biases to the table and often judge based on first impressions. Not cool. But there's a fix for this too! Take a page from Google's book:

1. Break It Down: Split the decision into smaller parts. This helps keep everyone focused on the important stuff.
2. Ask the Right Questions: Have a set of questions ready about how candidates act in different situations.
3. Team Decision: Instead of relying on one person's gut feeling, get a bunch of people together to review all the info collected. Then, decide as a group.

The Complexity of Eliminating Noise

Eliminating all noise isn't always worth the effort. The costs can be too high compared to the benefits. One major concern is fairness because machines can't replace human judgment, especially in critical situations. It's also expensive for institutions like schools to handle.

Reducing noise can lead to more problems than it solves. For instance, while algorithms can make accurate decisions without interference, they can introduce biases that aren't acceptable. People trust human judgment because it's thoughtful and considers moral values that no one wants to ignore. For example, mercy is a human quality that algorithms shouldn't override. If the methods used to reduce noise are unfair or simplistic, the solution is to develop better methods instead of ignoring the issue.

Workplace Standards and Judgment

Social values change over time, and being open to different viewpoints can help new ideas to emerge. In workplaces, strict rules can stifle creativity and make tasks feel less human. Having more flexible standards can help reduce unnecessary constraints.

When it comes to standards, which can be open to interpretation, it's important to cut out unnecessary complexity. Standards are purposely vague to allow for thoughtful consideration. For instance, a university might have a standard policy on sexual harassment, but it won't cover every possible scenario. When making decisions, remember that accuracy is key, not just expressing yourself.