What’s the key to personal progress? Plan like a pessimist, dream like an optimist, says a new book

Optimism and pessimism are so hard to deal with.

Pessimism is more intellectually seductive than optimism and captures more of our attention. It’s vital for survival, helping us prepare for risks before they arrive. But optimism is equally essential. The belief that things can be, and will be, better even when the evidence is murky is one of the most essential parts of everything from maintaining a sound relationship to making a long-term investment.

A big thing to know about how people think is that progress requires optimism and pessimism to coexist. They seem like conflicting mindsets, so it’s more common for people to prefer one or the other. But knowing how to balance the two has always been, and always will be, one of life’s most important skills.

The best financial plan is to save like a pessimist and invest like an optimist. That idea – the belief that things will get better mixed with the reality that the path between now and then will be a continuous chain of setback, disappointment, surprise, and shock – shows up all over history, in all areas of life.

John McCain became the most famous Vietnam prisoner of war. But at the time, Admiral Jim Stockdale was the highest-ranking POW. Stockdale was tortured routinely, and at one point attempted suicide out of fear he might break and give up sensitive military information. Decades after he was released, Stockdale was asked in an interview about how depressing life in prison must have been. He pushed back and said, actually, it was never depressing at all. He never lost faith that he would prevail – that he’d be released and reunited with his family.

Pure optimism, it seems.

Right?

Not really.

Stockdale was then asked who had the hardest time in prison. He said that was easy: “It was the optimists.” The prisoners who constantly said, “We’re going to be home by Christmas” were the ones whose spirits were shattered when another Christmas came and went. “They died of a broken heart,” Stockdale said. There is a balance, he said, between needing unwavering faith that things will get better while accepting the reality of brutal facts, whatever they may be.

Things will eventually get better. But we’re not going home by Christmas.

That’s the balance – planning like a pessimist and dreaming like an optimist. That mix is counterintuitive, but it’s so powerful when done right. Remaining optimistic while accepting the reality of despair is fascinating to witness.

“The American dream” was a phrase first used by author James Truslow Adams in his 1931 book The Epic of America. The timing is interesting, isn’t it? It’s hard to think of a year when the dream looked more broken than in 1931. When Adams wrote that “a man by applying himself, by using the talents he has, by acquiring the necessary skills, can rise from lower to higher status, and that his family can rise with him,” the unemployment rate was nearly 25 per cent and wealth inequality was near the highest it had been in American history. When he wrote of “that American dream of a better, richer, and happier life for all our citizens of every rank,” food riots were breaking out across the country as the Great Depression ripped the economy to shreds. When he wrote of “being able to grow to fullest development as men and women, unhampered by the barriers which had slowly been erected in older civilisations,” schools were segregated and some states required literacy tests to vote.

At a few points in American history, the idea of the American dream looked so false, so out of touch with the reality everyone faced. Yet Adams’s book surged in popularity. An optimistic phrase born during a dark period in American history became an overnight household motto. One-quarter of Americans being out of work in 1931 didn’t ruin the idea of the American Dream. The stock market falling 89 per cent – and bread lines across the country – didn’t, either. The American Dream actually may have gained popularity because things were so dire.

You didn’t have to see the American Dream to believe in it – and thank goodness, because in 1931 there was nothing to see. You just had to believe it was possible and then, boom, you felt a little better.

Psychologists Lauren Alloy and Lyn Yvonne Abramson have a theory I love called depressive realism. It’s the idea that depressed people have a more accurate view of the world because they’re more realistic about how risky and fragile life is. The opposite of depressive realism is “blissfully unaware.” It’s what many of us suffer from. But we don’t actually suffer from it, because it feels great. And the fact that it feels good is the fuel we need to wake up and keep working even when the world around us can be objectively awful, and pessimism abounds.

Excerpted with permission from Same as Ever: Timeless Lessons on Risk, Opportunity and Living a Good Life, Morgan Housel, Harriman House.