Do forecasts have any value?

Eighteen months ago, not a lot of analysts were predicting that the oil price would fall below $30 a barrel or that the rand would be trading at around R16.50 to the dollar.

Most likely, this is because humans are prone to extrapolate the recent past. In July 2014, when Brent-crude was trading at roughly $105 and the rand at R10.50, the current situation may have seemed highly improbable.

Which bodes the question: Does forecasting have any value in a world where things change so quickly?

Speaking to Moneyweb on a recent visit to South Africa, Michael Mauboussin, Wall Street investment strategist and head of Global Financial Strategies at Credit Suisse, said forecasting generally does have value, but in dynamic environments there are no simple answers.

Mauboussin drew on the findings of Phil Tetlock, professor of psychology and political science at the University of Pennsylvania, who examined the predictions of various experts over an extended time period. His findings suggest that the predictions of the average professional are not much more than guess work. However, his research also indicates that there is a small group of “superforecasters” who are consistently better at forecasting than their average counterparts.

One of the key attributes of these superforecasters is that they are constantly updating their views as new information becomes available.

“One of the biggest challenges is to constantly keep your mind open and to update points of view,” Mauboussin said.

When it comes to investing, the key challenge is to find gaps between what asset prices represent and what is likely to occur.

To illustrate his point, Mauboussin used horseracing as an example. When attending a horse race there are odds on the board, which represents a set of expectations about how the horses will perform, but really making money will require figuring out which horses are mispriced.

“Knowing which horse will win doesn’t make you money if it is all priced into the odds. So it is really this notion of odds – performance versus expectations – that is crucial.”

Literature on forecasting also suggests that people tend to be overconfident in their views – often because they extrapolate the current reality.

He said even if you ask people to define a range of possible outcomes for the price of oil in a year’s time, most people would provide ranges that are too narrow when left to their own devices.

Profile of a superforecaster

There is a vital distinction between IQ (intelligence) and RQ (rationality quotient) – the ability to make good decisions, Mauboussin said.

While there is not much that can be done about someone’s IQ, it is possible to cultivate RQ and to improve your thinking about the world and to make better decisions.

Superforecasters are actively open-minded. While this sounds fairly straightforward, it is quite difficult to apply in practice, in part because once someone has a set of beliefs, it is much easier to preserve the beliefs than to refine them, Mauboussin said.

While working in teams could improve the accuracy of forecasts, there is nothing “magical” about teams, Mauboussin said.

Teams have to be the right size (typically the optimal number is four to six people) and have the right composition with the emphasis on cognitive diversity.

Teams also have to be managed properly. Teams that fail tend to be those that don’t consider as many alternative points of view as they should. The best team leaders persuade their team members to share different points of view, which can then be vetted, he said.

But does the same premise apply in a market like South Africa, which is highly sensitive to political developments?

In some markets the level of predictability is quite high, but as more variables are introduced – economic or political – it becomes more challenging, Mauboussin said.

It is also important to realise that investing is a game of probabilities. Over the long haul equity markets go up, but in any particular year the performance is probabilistic.

“It is a very uncomfortable thing to live with often. You have to train yourself to live with it.”

In the South African environment forecasting may be more difficult than in other markets if there is political change or because companies may be more exposed to commodities, but the same basic principles apply, Mauboussin said.

But how does one marry suggestions that the world is shaped by Black Swan events – events that are not anticipated, consequential or explained after the fact as coined by Nassim Taleb – with suggestions that forecasting has value and that these skills can be refined?

Taleb questioned whether these superforecasters would be any better than anybody else at anticipating Black Swan events and Tetlock immediately conceded that point, Mauboussin said.

The question then becomes if only Black Swan events shape the course of world events?

Mauboussin said while making forecasts about the oil price or the exchange rate do not fall into the purview of Black Swans, it might still be useful to get a probabilistic assessment.

“I think it is a bit of an extreme stance to say none of this stuff is useful. I think it is actually very useful.”

However, one needs to be mindful that these Black Swan events do occur.

Tetlock also noted that some of the events that are considered Black Swans are actually Grey Swans – the distribution of these events are roughly known, although the timing is unknown, Mauboussin said.