A Simple Way to Avoid Being Wrong

Not Even Warren Buffett Could Predict the iPhone Would Decrease Chewing Gum Sales is a fun narrative that I almost wrote about.

But it’s probably wrong.

So I wanted to know why I almost made that mistake and also how to prevent making the same mistake in the future.

Buffett did a great Q&A that that I’ve written about before. In one part while talking about moats he makes a prediction:

I’m not going to be able to figure out what the moat’s gonna look like for Oracle or Lotus or Microsoft 10 years from now. [Bill] Gates is the best businessman I’ve ever run into, and they’ve got a helluva position, but I really don’t know what their business is gonna look like 10 years from now. And I certainly don’t know what his competitors’ businesses are gonna look like 10 years from now… I know what the chewing gum business is gonna look like 10 years from now. I mean, the internet is not gonna change how we chew gum.

And then just a few months ago a bunch of places (including Business Insider and Recode) wrote a post with this chart:

Made by Euromonitor International

And they all basically said the same thing: “We’re so addicted to our phones that we don’t look up to buy chewing gum at the grocery store checkout. The iPhone has changed everything!

At a quick glance the idea seems very reasonable, even intuitive.

We know how much the iPhone has changed everything else about our lives, we can see ourselves bored on our phones during checkout, it feels a simple narrative, etc.

Until you think about disconfirming evidence.

What could prove this idea is wrong?

Everything I said about this being easy to believe is all evidence that confirms the idea.1

But what about something that would disprove it?

I can think of 2 simple things that would do the trick:

  • Is gum mainly bought at checkouts? If not, people being bored in checkouts would have to be a really strong cause to affect sales
  • Have sales of other products primarily available at checkouts also declined? I.e. why would distracted shoppers not buy gum because of their phone but continue to buy, say, candy?

It’s “hard” to think of disconfirming evidence when you’re first looking at an idea. It takes a concerted and conscious effort.

I found a simple study in the Robot’s Rebellion that confirms this.

Each of the boxes below represents a card lying on a table:

  • Each one of the cards has a letter on one side and a number on the other side.
  • Is the following rule true: If a card has a vowel on its letter side, then it has an even number on its number side.
  • Which card or cards must be turned over in order to find out whether the rule is true or false?

Take a moment to think about this before reading on.

The K is easily ignored and the A is simple. But the 8 and 5 are tricky.

The right answer feels like 8, and that’s what most people choose. But it’s wrong.

Picking the 8 comes from a desire to seek confirming evidence

Which is why we tend to ignore the 5. We already know it’s odd, so it can’t confirm the other side is a vowel.

But it could prove the rule to be wrong, i.e. disconfirming evidence.

If we turn it over and the letter is a vowel, we know the rule is wrong.

Seeking disconfirming evidence takes effort. We’re all biased to interpret new evidence according to our previous knowledge of the world.

Even stopping for a few seconds to ask “what would prove this false” would be a huge improvement in the project of finding truth.

  1. Confirmation bias FTW