Chapter 11: Too Sure of Ourselves

One Weekend, Five Weekends

Inês is thirty-seven and has decided to repaint her kitchen. She looks at the walls on a Thursday evening and makes a plan. On Saturday morning, she will buy the paint and brushes. On Saturday afternoon, she will move the furniture, cover the floor and paint the first coat. On Sunday, she will paint the second coat and put everything back. By Sunday evening, she will be cooking dinner in a fresh, bright kitchen. She tells her sister about the plan on the phone. "One weekend," she says. "Easy."

On Saturday morning, the shop does not have the colour she wanted, and choosing another takes an hour. At home, she discovers that the old paint is peeling in several places and must be scraped off first. Moving the refrigerator reveals a patch of damp that needs to dry. The first coat takes much longer than expected, because the corners and edges need a small brush. On Sunday, she has guests for lunch that she had forgotten about. By Sunday evening, one wall is finished.

The kitchen is finally done five weekends later. It looks lovely, and Inês is proud of it. But she is also a little puzzled. She is not new to this kind of work. She painted her living room two years ago, and that also took far longer than she planned. Why did she believe, once again, that it would take one weekend?

Inês's experience is shared by almost everyone who has ever planned anything: a home repair, a piece of writing, a journey, a wedding, a move, a school project or a new business. We expect our own plans to go faster, more smoothly and more cheaply than they usually do, even when we have seen similar plans take much longer before. This chapter is about this pattern and about a wider family of patterns to which it belongs: our tendency to be more sure of ourselves than the facts justify.

Three Kinds of Overconfidence

Researchers use the word overconfidence for being more sure than the facts justify. It is one of the most widely observed patterns in the study of judgment. But it helps to see that it comes in at least three different forms.

The first form is thinking we are better than we are. We may overestimate how quickly we can finish a task, how much we can remember, how well we can do in a test or how much control we have over events. Inês overestimated how quickly she could paint a kitchen. A student who is sure she will remember everything without notes, or a driver who is sure he can drive safely while tired, is showing the same pattern.

The second form is thinking we are better than other people. When people are asked to compare themselves with others on qualities that they value, such as driving skill, kindness, honesty or sense of humour, most tend to place themselves above the middle. Of course, not everyone can be above the middle. This pattern is stronger for easy, everyday skills and for qualities that are vague and hard to measure. Interestingly, for tasks that are clearly very difficult, such as juggling or writing computer programs for someone without training, people often place themselves below average instead. So this form of overconfidence depends on the kind of task.

The third form is being too sure that our knowledge is precise. In a common kind of study, people are asked questions about numbers they do not know exactly, such as the year a historical event happened or the length of a river. Instead of a single answer, they are asked to give a range that they are almost certain contains the true answer. If people were well calibrated, meaning that their confidence matched their accuracy, the true answer would almost always fall inside their ranges. In practice, the true answer falls outside people's ranges much more often than they expect. Their ranges are too narrow. They feel more certain than their knowledge allows.

Of the three forms, the third seems to be the most consistent. People in many different groups, including experts in their own fields, tend to give ranges that are too narrow. This matters because many real decisions depend on knowing how uncertain we are. A person who knows that a project might take between one and five weekends will plan very differently from a person who is sure it will take one.

The Planning Fallacy

Inês's painting plan is an example of a pattern that has its own name: the planning fallacy. The planning fallacy is the tendency to expect our own plans to go faster and more smoothly than similar plans usually do. A fallacy is a mistake in reasoning, and this one is remarkably common.

In one well-known kind of study, students writing a long piece of work for their studies were asked to predict when they would finish it. They were also asked when they would finish if everything went as well as possible, and when they would finish if everything went as badly as possible. Many of the students took longer than their predictions, and a good number took longer even than their own "worst case" predictions. The students were not lazy or careless. They simply imagined the work going roughly as planned, and real work rarely does.

The same pattern appears in much larger projects. Building projects, public works and large organizational changes very often take longer and cost more than planned. This happens so regularly that people who study such projects treat it as a normal feature rather than a rare accident.

One of the most interesting things about the planning fallacy is that it applies mainly to our own plans. When people predict how long someone else's project will take, they are often more realistic, and sometimes even too pessimistic. And people often know perfectly well that their past projects took longer than planned. Yet when they plan the next one, they somehow believe that this time will be different. Inês knew that her living room had taken much longer than expected. She still planned the kitchen for one weekend.

Part of the reason is that we explain away our past delays. When we remember a project that took too long, we tend to think of the particular reasons: the shop was out of paint, the weather was bad, a child was ill. Each reason seems like a one-time accident that will not happen again. What we miss is the general lesson, which is that some accident always happens. The particular surprises change from project to project. The fact that there are surprises does not.

The Inside View and the Outside View

Why does this happen? One helpful explanation is the difference between two ways of looking at a plan.

The first way is the inside view. When we plan from the inside, we think about our own project in detail. We imagine the steps one after another: buy the paint, move the furniture, paint the first coat, paint the second coat. We picture ourselves working through them. This feels like careful planning, and in a sense it is. But it has a serious weakness. The steps we imagine are the steps that go well. We do not imagine the shop being out of the right colour, the peeling paint, the damp patch or the forgotten guests, because we do not know about them yet. Each surprise on its own is unlikely. But in almost every real project, some surprises happen. The inside view leaves no room for them.

The second way is the outside view. When we plan from the outside, we set aside the details of our own project for a moment and ask a different question: how long do projects like this usually take, for people like me? Instead of imagining our own steps, we look at a group of similar cases, sometimes called a reference class. "When I and people I know have painted a room before, how long did it take?" "When students in my course write this kind of essay, how long do they usually need?" "When people open a small shop like this, how long does it usually take before it earns enough to pay its costs?"

The outside view usually gives a more realistic answer, because the similar cases already contain all the surprises that the inside view leaves out. Nobody planned the damp patches, broken tools and forgotten appointments in those other projects either, but they happened, and they are included in how long the projects actually took.

The best planning often combines both views. Start with the outside view to get a realistic baseline: projects like this usually take about this long. Then use the inside view to adjust a little for what is truly special about your situation. If you are unusually experienced or have more help than most people, you might expect to be a little faster than the baseline. But be careful. Most people believe their situation is special, and most of the time it is not as special as it feels.

Looking Back With Too Much Certainty

There is a close relative of overconfidence that works backwards in time. After something has happened, we often feel that we knew it would happen all along. This is called hindsight bias.

When a sports team loses an important match, many fans say afterwards that they could see the defeat coming. When a relationship ends, friends may say that the signs were obvious. When a business fails, people point out the mistakes that, looking back, seem clear. In studies, people who are told the outcome of an event often remember their earlier predictions as closer to that outcome than they really were.

Hindsight bias matters for decisions because it makes the world seem more predictable than it is. If we believe we "knew it all along" about past events, we will tend to feel more confident about predicting future ones. It can also make us unfair to ourselves and to others. A decision that was reasonable at the time, given what was known, can look foolish afterwards only because we now know how it turned out.

One simple protection is to write down your predictions and your reasons before you know the outcome. Later, when you look back, you will see what you really thought, rather than what your memory tells you that you thought. This habit, which we have already met in the decision notebook from the first chapter, is one of the best ways to learn from experience honestly.

Hindsight bias also affects how we judge other people's decisions. A doctor, a teacher, a parent or a manager may be blamed for a decision that turned out badly, even if it was a sensible decision based on what anyone could have known at the time. A fairer question is not "Did it turn out well?" but "Was it a good decision, given what was known and the time available?" Good decisions sometimes lead to bad results because of bad luck, and poor decisions sometimes lead to good results because of good luck. Judging only by results teaches the wrong lessons.

The Good Side of Confidence

It would be wrong to suggest that confidence is simply a problem. A certain amount of optimism and self-belief is valuable. People who expect things to go well are often more willing to start difficult projects, to keep going when they meet obstacles and to recover from setbacks. Many of the most useful things humans have ever built were begun by people who were somewhat too optimistic about how easy they would be.

Confidence also matters socially. People who speak with reasonable confidence are often listened to more, and a team that believes in its project may work harder on it. And for many people, the problem is not too much confidence but too little: they doubt themselves so much that they do not try things they could do well.

The aim, then, is not to stop being confident. It is to be confident in the right way. One helpful idea is to separate two kinds of confidence. The first is confidence in yourself as a person: the belief that you can learn, cope and try again. This kind of confidence is almost always helpful. The second is confidence in particular predictions: that this task will take one weekend, that this plan will work, that this number is correct. This second kind should be held more lightly, checked against the outside view and adjusted when new information arrives. You can believe strongly in your ability to finish the kitchen while also admitting that it might take five weekends.

There is also a social side to this. When you share a plan with others, the way you describe your confidence affects what they do. A colleague who hears "This will definitely be ready on Friday" will plan around Friday. A colleague who hears "I am aiming for Friday, but it could be Tuesday" can plan more wisely. Sharing an honest range is not a sign of weakness. It is a kindness to the people who depend on your predictions.

Three Lives and Their Plans

Abdullah is fifty-five and has run a successful small bakery for many years. He decides to open a second bakery on the other side of town. He has a detailed plan: the new shop will be ready in two months and will be busy within three. A friend who has helped several small businesses asks him a simple question: "Of the shops you know that opened a second branch, how many were ready in two months and busy within three?" Abdullah thinks and realizes that he cannot name one. Most took longer, and some struggled for a year. He does not give up his plan, but he changes it. He keeps more money in reserve, plans for the new shop to take six months to become busy and arranges for an experienced worker to run the first bakery while he is away. The second bakery takes eight months to become busy, but because he planned for a longer wait, it survives.

Zhu is twenty and preparing for an important exam. She is sure she understands the material well, because when she reads her notes, everything looks familiar. Her study partner suggests testing each other without notes. Zhu is shocked to find how much she cannot recall. The feeling of familiarity, she realizes, was not the same as being able to use the knowledge. Researchers who study learning have found this pattern often: rereading notes can create a feeling of knowing that is stronger than real knowledge, while testing oneself gives a more honest picture. Zhu changes her study plan to include regular self-testing, and her confidence becomes more accurate.

Paulo is sixty-eight and is building a small wooden shed in his garden. He has built several things before, and he remembers that each one took about twice as long as he expected. So this time he uses his own past as his outside view. He makes his usual detailed plan, which says three weeks, and then doubles it. He tells his wife six weeks. The shed takes seven. Paulo is pleased. "Much closer than usual," he says. His simple rule did not make his plan perfect, but it made it far more realistic than the inside view alone.

For all three of them, confidence was not the enemy. Abdullah, Zhu and Paulo all needed confidence to start. What helped them was a way of checking their confidence against something outside their own hopeful imagination: other shops, a self-test and a record of past projects.

Practical Steps

First, take the outside view. Before you estimate how long, how much or how well, ask: how did similar projects go, for me or for people like me? If you do not know, ask someone who has done it, or look for reliable public information. Use that as your starting point.

Second, break the task into parts. Large tasks are easy to underestimate because we picture them as a whole. Write down every step, including small ones such as buying materials, cleaning up and waiting for things to dry or for people to reply. Estimate each part separately and add them. This often gives a larger and more realistic total.

Third, give ranges, not single numbers. Instead of "one weekend", say "between one and four weekends, most likely two." A range reminds you and others that the future is uncertain. If your range feels very narrow, widen it.

Fourth, add a buffer. Once you have your estimate, add extra time and money for surprises. A common simple approach is to add a fixed share to every estimate, based on how your past projects went. If your projects usually take about twice as long as planned, as Paulo's did, then double your plan.

Fifth, test yourself honestly. When you feel sure that you know something or can do something, find a way to check it: try without notes, ask someone to quiz you or practise in a small, safe way first. A feeling of confidence is a starting point, not proof.

Sixth, keep score. Write down your predictions, with your reasons, before the outcome is known. Later, compare them with what happened. Over time, you will learn how well calibrated you are, and in which areas your confidence tends to run ahead of the facts.

A Short Exercise: The Prediction Diary

This exercise helps you discover your own personal planning pattern. It takes a few minutes each day for two weeks.

Each morning, choose two or three tasks that you plan to do that day. They can be anything: writing a report, cleaning a room, cooking a meal, answering emails, fixing something, travelling somewhere. For each task, write down how long you expect it to take. Then, next to that, write a range: the shortest time and the longest time you think it could reasonably take.

Each evening, write down how long each task actually took, and whether it fell inside your range. If it took longer, write one or two words about why: interruptions, an unexpected problem, a step you forgot, tiredness.

At the end of two weeks, look at your diary. First, count how often the actual time was longer than your main estimate, and how often it was shorter. Many people find that most of their tasks took longer. Second, count how often the actual time fell outside your range. If this happened often, your ranges were too narrow. Third, look at the reasons you wrote. Do the same reasons appear again and again? Perhaps you always forget travel time, or you are often interrupted at a certain time of day.

Now work out a simple personal rule. If your tasks usually took about one and a half times as long as you expected, you might add half to every future estimate. If one reason appears often, you might plan for it directly, for example by adding travel time to every appointment. Write your rule on a card and use it for your next important plan.

Many people find this exercise both humbling and freeing. It is humbling because it shows how optimistic our quick plans are. It is freeing because, once you know your pattern, you can plan more kindly for yourself, with fewer rushed evenings and fewer broken promises to others.

Questions People Often Ask

"If I plan for things to take longer, will I just work more slowly?" This is a reasonable worry. Some people do tend to fill whatever time they are given. One way to avoid this is to keep two numbers: a target for yourself, which can be ambitious, and a realistic estimate for planning and for promises to others. Work toward the target, but plan with the estimate. That way, you keep your motivation without letting others down when the surprises arrive.

"Are experts less overconfident?" In some areas, yes. Experts who get quick, clear feedback on their predictions, such as people who forecast the weather, tend to become well calibrated over time, because they see their mistakes every day. In areas where feedback is slow, rare or unclear, experts can be just as overconfident as anyone else, and sometimes more so, because their knowledge makes them feel sure. So it is wise to ask not only whether someone is an expert, but whether their field gives them regular chances to learn from their errors.

"I am not overconfident. I doubt myself all the time. Is this chapter for me?" Yes, in a different way. People who doubt themselves often still underestimate how long tasks will take, so the planning tools in this chapter can help them too. And the distinction between confidence in yourself and confidence in particular predictions may be especially useful. You can hold your predictions lightly while still believing that you are capable of learning, coping and trying again. If self-doubt is causing you great distress, talking with someone you trust or with a qualified professional can help.

Chapter Summary

Overconfidence means being more sure than the facts justify. It takes three forms: thinking we are better than we are, thinking we are better than others and being too sure that our knowledge is precise. The third form, giving ranges that are too narrow, is especially consistent. A related pattern is hindsight bias, the feeling that we knew something all along once we know how it turned out.

The planning fallacy is the tendency to expect our own plans to go faster and more smoothly than similar plans usually do. It comes from the inside view, which imagines our own steps going well and leaves no room for surprises. The outside view, which asks how similar projects usually go, gives a more realistic starting point.

Confidence in yourself is valuable; confidence in particular predictions should be held lightly. Taking the outside view, breaking tasks into parts, giving ranges, adding buffers, testing yourself and keeping score make plans more realistic. In the next chapter, we look at how our beliefs shape what we notice, and why we so often see what we expect to see.