* An expert on experts tells how to spot the bad ones

In his documentary, The Trouble with Experts, filmmaker Josh Freed takes on the growing number of experts eager to tell us how to live.

Is this documentary your revenge for some bum advice?
After the 2008 stock crash, I was down and everyone I knew was down 50 per cent. Obviously my financial expert and all the others didn’t know a lot more than we did. I began to wonder about all the other kinds of experts, and I started doing research. I was stunned. We spend a lot of time and money listening to people whom we think know more than they do. For experts, it’s easy to predict the obvious, but it’s hard for them to predict when something goes really wrong.

How likely are experts to be wrong?
The science of it is pretty spectacular. Science writer David Freedman, author of Wrong, determined that two-thirds of studies in major science journals are shown later to be wrong. Almost 100 per cent of long-term economic studies are proven to be wrong.
And in the granddaddy of all studies, Philip Tetlock, a psychology professor at the University of California, Berkeley, spent 20 years following 300 elite media and government experts making 82,000 predictions. After 20 years, the predictions did a fraction better than a chimpanzee throwing darts at a board.
Tetlock found that the best experts were uncertain, because that kept them thinking about it. But that expert you rarely see on television. They’re boring.

Which field is the most rife with bad experts?
That’s tough to say. I think maybe health. There are a million of those so-called nutrition and diet experts who claim to have specific foods or cures to help you live forever. Many of them are self-made experts who belong to some association. Ben Goldacre (the British doctor who writes yjr weekly Bad Science column in The Guardian) applied for membership in a nutrition consultants’ association in the name of his dead cat, Henrietta. He now has a fancy certificate saying Henrietta is a member.

You call wine experts the snootiest. How incorrect are they?
In France, we met legendary wine researcher Frederic Brochet who has been testing wine experts for decades. We watched as he swapped $500-a-bottle wine and $30-a-bottle wine. The tasters pronounced the wine in the great bottle a great wine, although it was the cheap wine.
He also had 57 wine experts taste two red wines, except that one was really a white wine dyed with food colouring. Not one expert noticed the difference. He calls it “grape expectations,” what you expect to taste.

What about economic pundits? 
The economy has always been hard to predict. There’s a famous quote by a Yale economics professor: “Stocks have reached what looks like a permanently high plateau” made on Oct. 17, 1929, days before the Great Crash.
We visited an ex-management consultant who flew all over the world advising top companies and governments. He says the most important thing is to look right, dress right and drop a lot of jargon. No one knows with statistical accuracy how to make a business run or predict the economy.
The guy who really struck me is William White. He helped run the Bank of Canada and was a top banker in Europe. He said that we pretend economics is a science, but it’s not. It’s mass behaviour. It’s unpredictable.

Experts are the new high priests, according to your documentary. Why now?
It’s partly the tremendous amount of information out there. You have thousands of decisions to make that are overwhelming. And it’s 24/7 television. Nothing is cheaper than a couple of experts debating.
We’ve always had people telling us the future, be it fortunetellers or oracles reading chicken entrails. Today we’re too educated to read chicken entrails, so we go to experts. We use Power Point instead of poultry.

How do you become an expert? Don’t you need degrees, stamps of approval, some vetting?
There are endless books on how to become an expert and a lot of courses. We went to a course that manufactures experts. The class was made up of respectable people, doctors, lawyers, fitness trainers. The course promised to make them look and sound like experts so they could go on television. The central message was to sound certain. Never use the word maybe. Always say always.

Life is complicated. We need expert advice. How do you tell a wise advisor?
The one who seems uncertain but offers ideas on how you can think about something. He’s not promising to save you but has five things that might help.
Now I trust most — bizarrely it’s the one we complain about most — Environment Canada weather forecasts. They give percentages — a 60 per cent chance of rain. They are statistically right more often than not.

How do you now get financial advice?
I do it differently now. I treat my financial advisor like someone you chat with, just one more voice in the wilderness. I get his opinion and others. I do a lot more work, more reading.

So now you’re an expert on experts. Why should I believe you?
I started as a regular person and now I’m practically an expert. The ultimate message is use your own brain and take more responsibility. There are no gods out there.