Gary Klein: Insights For Making Better Decisions
Guest Gary Klein, a research psychologist, discusses naturalistic decision-making, the difference between experience and expertise, and how to gain insights. He shares methods like pre-mortems, shadowboxing, and decision journals to reduce errors and accelerate learning.
Deep Dive Analysis
18 Topic Outline
Introduction to Gary Klein and Naturalistic Decision Making
Pathways to Insight: Connection, Contradiction, and Correction
Organizational Barriers to Insight and Innovation
Individual and Organizational Strategies to Foster Insights
Distinguishing Experience from Expertise and Identifying Experts
Evaluating Decision-Making and the Use of Decision Journals
Defining Mental Models and Their Four Key Components
The Role of Stories in Learning and Overcoming Language Limitations
Overcoming Fixation Errors and Knowledge Shields
Breaking Through Stagnation to Achieve Mastery
Cognitive Flexibility Theory: Adapting to Unexpected Situations
The Pre-mortem Technique for Reducing Project Errors
Critique of Cognitive Biases and the Value of Heuristics
Team vs. Individual Decision-Making Strategies
Improving Group Decisions and Surfacing Unique Insights
The Zone of Indifference: When to Stop Deliberating
Shadowbox Method for Accelerated Decision Training
The Role of Environment in Decision-Making and Voluntary Compliance
7 Key Concepts
Insight
An insight is a realization that often contradicts existing beliefs, leading to surprise and a change in thinking. Insights can emerge from connecting disparate ideas, noticing contradictions in expectations, or correcting flawed beliefs that were causing a problem.
Knowledge Shields
Knowledge shields are a variety of techniques, used by both scientists and individuals, to explain away inconvenient data or anomalies. This tendency to dismiss unexpected information prevents people from exploring inconsistencies and potentially gaining new insights.
Fixation Error
Fixation error occurs when individuals get stuck on an initial impression or understanding of a situation, even when subsequent information or anomalies suggest that the initial assessment is incorrect. It is overcome by noticing and exploring anomalies rather than explaining them away.
Mental Model
A mental model is a comprehensive set of beliefs about how something works, including its components and their interactions. It also encompasses its limitations, potential failure points, effective workarounds, and an anticipation of where others might get confused.
Cognitive Flexibility Theory
This theory focuses on helping people achieve expertise by preventing them from rigidly adhering to routines and standard procedures. The goal is to cultivate a mindset where individuals are naturally adaptive and even enjoy improvising when unexpected situations arise.
Zone of Indifference
The zone of indifference describes a situation where two or more decision options have advantages and disadvantages that are so perfectly balanced that it becomes extremely difficult to choose between them. The paradox is that if the options are truly balanced, the specific choice made has little impact on the outcome.
Positive Heuristics
This concept suggests that many cognitive biases, often criticized for leading to errors, are actually valuable mental shortcuts that are generally useful for making decisions and generating insights. These heuristics are part of our learned experience and are essential for effective functioning, even if they are not foolproof.
11 Questions Answered
Experts are highly aware of their mistakes, which 'eat at them' until they figure out a way around them, unlike journeymen who often forget their mistakes. They also typically have years of experience, a strong track record, and a rich mental model.
Organizations often prioritize predictability and error reduction over insights, which are disorganizing and disruptive. They tend to stifle new ideas, especially when experienced senior members recall past failures, leading to risk aversion.
Individuals can adopt an 'insight stance' by becoming curious about unexpected things, celebrating small discoveries, and attuning themselves to insights they and others have, rather than recoiling from unfamiliarity.
Organizations can create mechanisms for reviewing prematurely rejected ideas, acknowledging that experienced people may be risk-averse due to past failures, and fostering a culture that values exploration over strict adherence to norms.
A decision journal should capture the decision itself, the primary and secondary goals, the key pieces of information used, and other people or teams affected by the decision, providing a record of one's thinking at the time.
An effective story involves some mystery or an unexpected event, making listeners curious about the outcome. The storyteller should enjoy telling stories, and good stories typically result in an insight or transformation for the listener.
Experience is simply years spent doing something, while expertise involves reflecting on what happened, learning from mistakes, and developing a richer mental model, rather than just accumulating unexamined experiences.
Providing vicarious experiences, such as stories or scenarios of turbulent or challenging situations, can help individuals learn much more quickly than simply having a smooth, uneventful period of experience.
Team decision-making can leverage collective wisdom, but it can also lead to pressure for consensus, which may stifle bold ideas or individual insights. The optimal strategy depends on the situation and context, sometimes requiring an autocratic switch under urgency.
Groups can improve by having individuals generate their ideas, interpretations, and assessments privately before sharing them. A leader should also genuinely query team members, being curious about differing thoughts rather than just checking boxes.
When facing a 'zone of indifference' where the advantages and disadvantages of two options are almost perfectly balanced, it's best to recognize that the specific choice doesn't matter significantly and pick one to save time and resources.
29 Actionable Insights
1. Cultivate Curiosity for Insights
To spark insights, cultivate curiosity about things that don’t make sense, rather than dismissing anomalies. This curiosity helps in investigating further and challenging existing beliefs.
2. Adopt an Insight Stance
As an individual, maximize insights by adopting an ‘insight stance’ that embraces new and unexpected information with curiosity. Actively celebrate your own small insights and those of others, rather than only dwelling on mistakes, to increase sensitivity to them.
3. Balance Error Reduction, Insight
To improve performance, focus not only on reducing errors but also on increasing insights. Most organizations neglect the latter, which is crucial for growth.
4. Unlearn to Achieve Mastery
To achieve mastery and break through performance plateaus, actively engage in unlearning. This involves identifying and letting go of flawed or limited conventions and beliefs that no longer serve broad applicability.
5. Embrace Turbulent Experiences
Seek out or value turbulent experiences in your career, as they provide a richer and more detailed learning surface area compared to smooth, low-stress periods. These challenges force adaptations, mistake identification, and recovery, accelerating expertise.
6. Notice Anomalies to Break Fixation
To break out of fixation errors, actively notice and explore anomalies that contradict your initial understanding. Avoid explaining away inconvenient data, as this leads to holding onto wrong impressions for too long.
7. Conduct a Project Pre-Mortem
Conduct a pre-mortem by imagining a project has failed in the future and then individually writing down all the reasons why. Share these reasons aloud, going around the room, to surface unforeseen flaws, diminish overconfidence, and foster a culture of candor and trust. Conclude by having each person write down personal actions to mitigate identified risks.
8. Build Trust in Encounters
Adopt a mindset in any encounter, especially with civilians or lawbreakers, to aim for increased trust by the end of the interaction. This shift from ‘make them comply’ to ‘build their faith in me’ governs behavior to create greater trust and voluntary cooperation.
9. Train with Shadowbox Method
Use the Shadowbox method for decision training: go through challenging scenarios, pausing at decision points to rank options/goals/information and state reasons. Compare your choices and reasoning to those of experts, focusing on their detailed rationale to learn to see the world through their eyes.
10. Journal Decisions for Reflection
Journaling decisions helps in reflection, distilling complex thoughts, and identifying gaps in understanding or areas for further information gathering. It also allows for reviewing past patterns in decision-making.
11. Structure Decision Journal Entries
When creating a decision journal, record: the decision itself, primary and secondary goals, key information used, and affected people/teams. This provides a valuable record for later reflection on overlooked cues or goals.
12. Provide Vicarious Experiences
To fast-track expertise for individuals with limited direct experience, provide vicarious experiences through stories or challenging scenarios. This allows them to learn from others’ crises and adaptations without having to live through them directly.
13. Use Stories to Share Abstractions
When learning from another person’s abstraction, ask for the stories behind it to overcome language limitations and miscommunication. Inquire about surprising events, what happened, why they were surprised, and how they made sense of it to gain a deeper understanding.
14. Elements of Effective Storytelling
To tell an effective story, include an unexpected mystery that engages the listener and makes them want to know the outcome. The storyteller should enjoy telling stories, and the narrative should ultimately lead to an insight or transformation.
15. Recognize Zone of Indifference
When faced with two nearly equally effective options (a ‘zone of indifference’), recognize that further deliberation is often unproductive. Pick one option, even by flipping a coin, and reallocate time to more fruitful endeavors.
16. Avoid Pre-Mortem Mistakes
Avoid common pre-mortem mistakes by framing it as ’the project has failed, explain why’ (not ‘what can go wrong’), ensuring rapid-fire contributions from each person, and having the leader actively participate first to foster trust.
17. Signal Unique Insights in Meetings
Improve meeting effectiveness by changing what people signal as valuable, encouraging unique insights rather than regurgitating known information. Ask participants to contribute something valuable and uniquely insightful to the problem.
18. Generate Ideas Privately First
To generate more insights in group settings, have individuals independently and privately generate their ideas and concepts first, before sharing them with the larger group. This approach is more effective than collective brainstorming.
19. Define Problems Individually First
Before making a group decision, have each person individually write down their definition of the problem being solved. Comparing these statements will reveal variance and help ensure the group is addressing the actual problem.
20. Leaders: Query Team Honestly
Leaders should honestly and curiously query team members for their thoughts and differing perspectives, rather than just checking boxes. This approach genuinely seeks out diverse ideas and helps capture a richer understanding of the situation or goals.
21. Ask About Surprises in Reviews
In project review meetings for complex or ‘wicked’ problems, ask team members what has surprised them since the last meeting. A lack of surprises is a red flag, as examining unexpected events can reveal crucial insights and prompt necessary adjustments to the problem definition or goals.
22. Avoid Consensus in High Stakes
Avoid consensus decision-making in high-stakes or dangerous environments, as it creates pressure to conform and can lead to suboptimal outcomes. Instead, use secret voting to allow individuals to express their true opinions without fear of repercussions.
23. Contextual Team Decisions
There is no universal best approach for team decision-making; the most effective strategy depends entirely on the specific situation and context.
24. Foster Voluntary Compliance
To improve decision-making in conflict situations, focus on structuring the environment to gain voluntary compliance rather than relying on intimidation. Study and adopt techniques used by ‘good strangers’ who are skilled at diffusing situations and creating benign environments.
25. Ask Experts About Mistakes
To identify an expert, ask them about their last mistake. Experts are aware of and reflect on their mistakes, unlike competent but non-expert individuals who tend to forget them.
26. Implement Idea Review Mechanism
Organizations serious about fostering insights should implement a review mechanism for ideas rejected prematurely. This prevents good, but fragile, ideas from being discarded by a single person in the chain of command.
27. Understand Cognitive Heuristics
Rather than focusing on ‘de-biasing,’ understand that cognitive biases are often heuristics that are generally useful, even if not perfect. Recognize their strengths and how they derive from experience, rather than solely focusing on their failures in specific contexts.
28. Video Shadowbox Training
For video-based Shadowbox training, watch a scenario, then rewatch it, pausing to highlight important cues and explain why. Compare your identified cues and reasoning with those of experts to learn their focus and thought processes.
29. Embrace Disruption for Flexibility
Develop cognitive flexibility by avoiding rigid routines and embracing unexpected challenges. Cultivate a mindset where you are excited, rather than frustrated, when plans go awry, viewing it as an opportunity to improvise and invent new solutions.
9 Key Quotes
Experts are well aware of their mistakes and their mistakes eat at them.
Gary Klein
Insights are disorganizing, as you point out. Insights make you change the way you think, make you change all kinds of things.
Gary Klein
Good ideas are fragile and precious and easily discarded.
Gary Klein
The act of journaling is a way of helping you inquire better and be curious about things that you haven't thought about.
Gary Klein
Stories are not hampered by the limitations of language because now we have an incident account that we can drive.
Gary Klein
The way we break out of fixation is we notice the anomalies. The way we get stuck in fixation and making fixation errors is we explain away the anomalies, hold on to the original wrong impression until it's far too late.
Gary Klein
People who break through and move to the next level are the ones who engage in unlearning, who realize there are certain conventions they've bought into or beliefs they hold that are either wrong or are limited and don't apply as broadly as they imagine.
Gary Klein
You don't want to lock in to our original ideas because almost certainly our original ideas are going to be inadequate.
Gary Klein
If the advantages and disadvantages of the two options are almost perfectly balanced, it doesn't matter which one we pick.
Gary Klein
4 Protocols
Decision Scorecard Method
Shane Parrish- Sit down with an employee to review major decisions made in the previous year (both employee and manager bring their lists).
- Compare notes on the identified decisions.
- Examine which decisions worked and which didn't, considering if good decisions had bad outcomes or bad decisions got lucky.
- Focus on understanding what the person was thinking at the time of the decision to learn from both successes and failures.
Pre-mortem Technique
Gary Klein- Conduct the premortem at the end of a project kickoff meeting, after the plan has been finalized.
- Instruct the team to relax and imagine it is six months (or a year) from now, and the project has gone off the rails and failed in a major way, with this failure being certain and infallible.
- Have everyone privately write down all the reasons why this project failed for two minutes.
- Go around the room, starting with the leader, and have each person state one reason for failure that hasn't been mentioned, writing them on a whiteboard. Continue around the room, potentially a second or third time, until all unique reasons are captured.
- Have everyone take another two minutes to privately write down what they can personally do to reduce the chance of these identified failures occurring.
Shadowbox Decision Training (Text-based)
Gary Klein- Create a tough, challenging scenario presented in text.
- Stop the scenario at various decision points.
- At each decision point, present multiple options (e.g., courses of action, goals, information to pursue) and have the participant rank order them, writing down their reasons.
- Immediately after, show the participant how their ranking compares to a panel of experts and display the experts' synthesized reasons (what they noticed, inferred, worried about, opportunities they saw).
- Continue through the scenario, repeating the process at subsequent decision points.
Shadowbox Decision Training (Video-based)
Gary Klein- Present a one or two-minute video of a situation (e.g., police body cam footage, staged hospital scene).
- Allow the participant to watch the scenario from beginning to end.
- Restart the scenario and allow the participant to pause it at any point they choose.
- When paused, the participant can highlight any cue they deem important by placing a circle around it and writing down why they highlighted that cue.
- After the participant finishes, show them the same scenario with expert annotations, revealing what experts focused on, when they focused on it, and the reasons they gave for their focus.