Just how accurate are oil price predictions?

“Economists don’t forecast because they know, they forecast because they’re asked.”

K. Galbraith, economist

The Wall Street Journal (WSJ) polls institutions every month on a range of economic variables including inflation, unemployment and West Texas Intermediate (WTI) crude oil prices. Each month, the survey asks for predictions for the forthcoming June and December. For the sake of consistency, I have reviewed the accuracy of forecasts made both six and 12 months prior to June and December each year. I reviewed surveys from mid-2007 to the end of 2016 and so this covered booms and busts, financial crises and quantitative easing, the Arab Spring and the shale revolution.

By means of a disclaimer, this is not an exhaustive study. By definition, it only covered a ten year period, and there is no guarantee that forecasters that were correct during this boom and bust period will be any more or less successful in future periods. It also says nothing about how well those same institutions did trying to predict other commodity prices including metal and agricultural prices. Finally, it only covers those forecasters that the WSJ surveyed – there may have been others who were more or less accurate in their predictions.

Using this data (available to view at materials-risk.com/crudeforecasts), I tried to answer the following three questions: were the forecasts correct?; were the predictions valuable?, and, third, was there a forecaster that you could have followed that would have led to a better overall result than taking the consensus? Let’s discuss each of these points in more detail.

First, were the forecasts right? The answer was clearly no. The average consensus forecast (ie, the average of the commodity price predictions) for WTI crude oil was off by 27% when forecasting six months out. Oil price forecasts looking twelve months out were only slightly worse, off by an average of 30%. Another way of looking at it is that only in three of the nineteen periods reviewed was the consensus six month forecast within 5% of the actual result.

Producers, manufacturers, investors and traders are making billions of worth of investment based on the outlook for commodity prices. If these forecasts are awry on as short term a time period as twelve months then how can they have confidence making decisions over much longer time periods?

Second, did any of the forecasts spot the major changes in the direction of the oil price over the past decade? In June 2008, WTI crude was trading at approximately $135 per barrel. The consensus prediction for December 2008 was just under $112 per barrel and $101 per barrel twelve months ahead. The reality was somewhat different. The financial crisis hit and with it the oil price was hit too. WTI crude prices fell to $41 per barrel in late December 2008, only rebounding to $70 per barrel in mid-2009. Almost all forecasters polled in mid-2008 saw prices falling over the next twelve months, but no one saw the scale of the collapse. The closest six month forecast, although over 50% higher than the outcome, came from Parsec Financial Management!

It was a similar story in trying to call the rebound in prices. Remember that oil and other commodities rebounded in 2009 as quantitative easing helped support prices. Back in December 2008, however, the consensus prediction for June 2009 was for prices to stay low, only nudging up from the current levels of the time. This time the consensus was over 30% too low. Only three forecasts called the market within 5%: Societe Generale, Barclays and the Economic and Revenue Forecast Council.

Over the next few years, oil prices traded in a gradually narrowing range between $70 and $110 per barrel. Sure enough, the consensus and individual forecasts, increasingly anchored against recent prices, turned out to be broadly correct – well at least within a range of 5–15%. Like many forecasters, these economists were driving with their eyes fixed on the rear view mirror, enabling them to tell us where things were but not where they were going. This bears out the old adage that “it’s difficult to make accurate predictions, especially with regard to the future.” The corollary is also true: predicting the past is a snap.

If we fast forward to June 2014, oil prices were trading at approximately $105 per barrel, having peaked at just over $112 per barrel ten months earlier. The consensus forecast was for oil prices to fall from those levels to below $99 per barrel in December 2014. In reality, the consensus proved too optimistic by 85%. All of the forecasters were over 65% too optimistic, apart from one – Parsec Financial Management had predicted oil prices to be in the late $60s per barrel range in December 2014, only 24% too high.

Does that mean that Parsec Financial Management have superior insight? Well, not quite. A look back through earlier forecasts reveals that they were consistently bearish all the way back to early 2010, calling for oil prices to stay around $50–70 per barrel, even though oil prices kept on rising. This is what’s known as the “stopped watch” method of prediction. If you keep on saying something extreme will happen and it eventually does then you are feted as a guru when, in reality, you were lucky (eventually) with the timing.

Predictions are most useful when they anticipate change. If you predict that something will stay the same and it doesn’t change, that prediction is unlikely to earn you much money or wow clients with your predictive abilities. However, predicting change can be very profitable for investors, while timing hedging strategies can be a welcome boost to both producers and manufacturers.

Third, was there a forecaster that you could have followed that would have led to a better overall result than taking the consensus? Of those 26 institutions that contributed prices for at least 14 of the 19 forecast periods and during the key turning points in the market identified above, three forecasters achieved a better than average result than the consensus when looking over a period of six months. These were: JP Morgan (26% forecast error, 4 correct calls); Comerica Bank (25%, 2) and The Conference Board (25%, 3). The most accurate institution achieved a two-percentage point improvement on the consensus, but still had an average six-month forecasting error of well over 20%. The research sample also includes Goldman Sachs, often famed for its supposed commodity prediction ability. How did they do? They were an average of 36% off with one correct call.

“The lucky idiot”

Nevertheless, the ability to predict, over very short periods says little about the quality of a forecast. As with investment success in the stock market, or anywhere else, it’s very difficult to measure success in retrospect or a priori. If someone you follow correctly predicts the price of oil, it looks good on paper, but what you can’t gleam from the magic number is what the risk was that it didn’t happen. To what extent was the forecaster a “lucky idiot”, as the investor Nassim Nicholas Taleb would call them?

That a forecast for future oil prices turns out right doesn’t mean it was bound to happen. Howard Marks, manager of Oaktree Capital, uses the example of the weatherman to explain:

He says there’s a 70 percent chance of rain tomorrow. It rains; was he right or wrong? Or it doesn’t rain; was he right or wrong? It’s impossible to assess the accuracy of probability estimates other than 0 and 100 except over a very large number of trials.

But the climate and the weather are never the same on any one day. A meteorologist can look back at similar patterns and infer what is likely to happen tomorrow. Commodity forecasters can do a similar exercise too, but it will never be the same. The world is always changing.

Risk exists only in the future, and it’s impossible to know for sure what it holds. No ambiguity is evident when we view the past. Only the things that happened, happened. That definiteness, however, doesn’t mean the process that creates outcomes is clear-cut and dependable. Many things could have happened, and the fact that only one happened devalues the variability that existed.

Remember, predictions are made with foresight, but tested with hindsight. It is easy to look back at a sequence of events that led to a forecast turning out correct and to lead the pundit, and anyone who had seen that forecast, to say: “I knew it would, it was obvious it would turn out that way.” The hindsight bias, as it’s known, prevents the forecaster and the consumer of that forecaster from reviewing whether it was correct because the pundit judged the risks correctly, or whether the pundit was just a “lucky idiot”.

The pundit (or lucky idiot) might be infamous for making one big call. But is that enough? Given enough events, even a monkey can make the right prediction eventually. Does that mean that the monkey is endowed with magical powers of insight about the future? Sadly no. For purely statistical reasons, outstanding performances tend to be followed by something less impressive. This is because most performances involve some randomness. On any given day, the worst observed outcomes will be incompetents having an unlucky day, and the best observed outcomes will be stars having a lucky day. Observe the same group on another day and, because luck rarely lasts, the former outliers will not be quite as bad, or as good, as they first seemed.

Regression to the mean, as it’s known, probably explains why many winners subsequently disappoint. And the disappointment will be spectacular if some people are taking bigger risks than others. The most impressive performance may combine skill with luck. In a financial market — or a casino — the easiest way to become an outlier is to make a big bet.

While randomness can explain much, hubris may also play a role. The economists Ulrike Malmendier of University of California, Berkeley and Geoffrey Tate of University of California, Los Angeles examined what happened to companies whose chief executives won accolades such as Forbes’s “Best Performing CEO” or BusinessWeek’s “Best Manager”. They picked a statistical control group of near-winners who might have been expected to win an award, but did not.

Like the near-winners, the winners ran large and profitable companies. However, those companies run by the winners did far worse in the three years following the award, lagging behind the near-winners by approximately 20%. The prizewinning CEOs, nevertheless, enjoyed millions of dollars more in pay. They were also more likely to write books, accept seats on other corporate boards and improve their golf handicap.

This article is based on a chapter from the book, Crude Forecasts: Predictions, Pundits & Profits in the Commodity Casino. Buy it to find out more about how you can make better predictions of future commodity markets and hold others to account.

Unlock commodity market insight now and subscribe to our email updates or

Everyday stable prices: How one entrepreneur is helping farmers plan for the future

“We’ve seen a halving in the number of UK dairy farms in the past ten years. This is a real problem.”

That’s Richard Counsell, entrepreneur and managing director of Stable, a business that aims to help farmers manage the impact of volatile commodity prices. In this conversation Richard shares how his background in software and farming provided the knowledge and insight to help farmers manage their risk.

Richard explains why giving traditional commodity futures markets are not equipped to really help ‘normal’ family farms in Europe and APAC manage risk effectively, but that having some form of risk management is crucial in giving farmer’s confidence to invest in new plant and machinery is so important. He shares how agricultural commodity prices are typically uncorrelated within a year and how providing this risk management is better delivered as insurance. Finally he explains why farming is potentially a great career opportunity of the future.

Stable is due to launch in early 2018. You can follow Richard on Twitter @richcounsell and Stable @stableprice. Also check out the resources on the bottom of this post that related to our conversation.

Richard Counsell (RC) “The first thing to say is that I am a farmer and so that is a massive advantage in terms of understanding what farmers are willing and able to take on board in this area. And as a kind of weird combination I previously built a software company in Chicago, and had this dual life of having a Somerset farming background. It was really the combination of this experience that came together really and the urgent need to do something to help family farms manage commodity price volatility.

But the agricultural sector in the US is very different from the UK isn’t it?

(RC) “When I sat down to have a beer with trader friends in Chicago and they explained futures and options it took me so long to get my head around it. How the hell would you ever transpose this into a European agricultural context where the farms are smaller and they don’t have this historical background in Futures and Options? I just kept thinking this is the wrong product for the wrong audience. Farmers aren’t traders, 90% of their time is spent worrying about yield and production. The opportunity cost of sitting down and wrapping your heads around derivatives is far too high for the vast majority of family-sized farmers.”

Agriculture in the UK and elsewhere in Europe is typically on a much smaller scale than in the US.

(RC) “Only 2.7% of farms are over 100 hectares in Europe. And then you go and talk to the risk management brokers and they say well unless you’ve got 400 hectares we won’t even pick up the phone. I totally get it from their perspective as it takes 2-3 months to get a farmer through the regulation side of things because of course farmers are regulated as retail clients and it’s not really worth their time and effort to get them through the process and even then it’s only really based around arable and there’s nothing there for the dairy or livestock farmers.”

Although futures markets have their historical routes in the US, as exchanges developed to help farmers manage risk, things have changed. Many farmers in the US complain that futures prices bear no resemblance to the underlying fundamentals, making it increasingly difficult to use futures markets for the purpose that they were originally intended.

(RC) “What I found fascinating when I was in America researching who uses futures, over 50% of the volume is short term speculation and so the basis risk is all over the place. If you are just trying to offset a bit of risk for your farm, it’s getting pretty complex with the levels of volatility that are going on. In 2015/16 the European press and politicians were saying futures exchanges are the answer but when you were over there and talked to normal sized farmers, they were telling me ‘don’t get your hopes up as it’s not a panacea.’ There’s some real problems here, because it’s changed from a simple risk transfer process into much more about short term speculation. To my mind it felt like it was shifting away from its historical roots and the real problem is was set up to solve.

So the challenge was to think how on earth could we offer an alternative and what would that look like? Because of my US software background, I was lucky enough to be able to call on some academic contacts from Harvard and University of Wellington in New Zealand and together we built a very simple options exchange, thinking it was perhaps the user experience that was the problem. But six months of development later and we started showing it to farmers and getting their feedback on what was effectively an agri-focused prediction market. It worked simply by asking a farmer, ‘Will the price of wheat be over £140 per tonne in September 2018?’ The farmer could just answer yes or no and then buy yes shares or no shares. I liked it because it had a very simple user experience, but in reality you are still asking the farmer to calculate the probability of a rise or fall…That was already too much.

At this point in the R&D I was doing talk at farming events all over the country and one speech that I did up in the Cotswolds was a real turning point for the initiative. I thought I was talking in pretty simple terms , but still using the language and terms of derivatives (i.e. puts, strikes) and it was honestly one of the most painful speeches I’ve ever given. Towards the end, one farmer stood up and said, ‘can I just ask one question…are you actually talking about me insuring my milk price?’ It was at that moment that I was like yes, yes it’s just insurance for your milk instead of your tractor. Then they all went, we totally get it, why didn’t you just explain it was insurance?” A lesson learned!

Sometimes it’s about breaking it down into a language that the audience is familiar with.

(RC) “It’s true, every word really really matters especially when you are in education mode and actually I would be a very old man before I educated British farmers about derivatives. Not because it’s too complex; farmers are some of the smartest people I know, just because the opportunity cost for busy farmers to learn about it is just too high. Crucially as well, we wanted to work with organisations that already had an existing financial relationship with farmers. The banks for example, have as much to gain from price insurance as the farmers as we make the farmers more creditworthy. The other benefit of selling insurance, is that 3rd parties can’t easily recommend a derivative product to a ‘retail’ farmer, as it’s got car crash written all over it. But with insurance you are suddenly back in the game of normal day to day business instead of having to get involved in all of the financial regulations and far more professionals can recommend a farmer comes to talk to us.

At this stage the Stable initiative was in an awkward position. The farmers were telling us that it had to be ‘real’ insurance and Insurance companies were rightly saying ‘commodities are systemic, you can’t insure systemic risk’. To overcome this I started researching new areas of insurance that might throw us a lifeline…areas such as Cat Bonds and Insurance Linked Securities for example. To help steer through this complex area, I turned once again to academia to help me make sense of it all.

I was introduced to Professor Hirbod Assa at Liverpool University. He works at the world renowned Institute of Financial and Actuarial Mathematics and they were like Rich you are onto something and it could potentially have huge impact. After over a year of collaborating with many of their academics, we figured out a ground breaking way to deal with the issue of systemic risk. The eventual solution was inspired by the A.G. Street adage, ‘Up corn, down horn’. Traditionally all farms were mixed and so farmers knew that if one crop tanked another would do well and so they were protected from large fluctuations in income. But as farms have increasingly specialised to seek economies of scale, that diversification benefit has largely been lost.

I challenged Liverpool to see if that phrase ‘Up corn, down horn’ is true, using the latest data science available i.e machine learning. To build an efficient platform, we needed to use an index, which meant we didn’t need to have an expensive claims process that would make the end product too expensive. As a farmer I pay a levy to the AHDB to collect pricing data on all the commodities that British farmers produce. And that valuable data just sits there and hardly anyone uses it. So then Liverpool University and our growing team of Quants and developers got stuck into the data and we ended up building a very sophisticated model and data driven platform that both forecasts prices for the individual commodities using a suite of sophisticated algorithms and then dynamically allocates capital across all the commodities in real time. In the UK alone we do 10^60 simulations and we’re rolling out to another 12 countries so you can start to see the size of the data science aspect of what we’ve built.

Instead of having to monitor a Bloomberg screen, we’ve essentially replaced that with three simple questions for a farmer to answer: ‘How much do you want to insure?’ i.e. 10 tonnes. ’How long do you want protection?’ i.e 1 year and finally ‘What price do you want protection from?’

The system generates a premium for the farmer and they can either accept or reject it, just like insuring your car (or perhaps tractor)!

But grain and other agricultural markets have been especially calm over recent years. Is there a risk that the premiums may not accurately cover the risk of an escalation in volatility?

(RC) “It’s not like trading on the markets with prices that change every day. We take the monthly calendar average from the AHDB prices, and it’s much less volatile than futures markets. As a farmer I’m much happier to use a monthly average so I don’t need to keep checking the position instead of doing my real job on the farm.

To reduce our risk the system also dynamically allocates capital. Say we had too many wheat farmers using the system; we would manage the availability of wheat insurance until sheep farmers, sugar or palm oil farmers joined. It’s an enormous correlation exercise. Put in the context of ‘Up corn down horn’ we’re acting like the world’s largest mixed farmer to hedge our own risk! There is almost no correlation between the price of Portuguese strawberries and Welsh lamb for example. The other challenge is from a regulation point of view we have to work very hard to ensure that only real farmers can participate and they can’t insure more than their farm could possibly produce for example.”

Having lots of uncorrelated markets to insure against is a potential goldmine for insurers, right?

(RC) Stable can offer risk that is very uncorrelated with cars, property etc., but what’s unique is that we also have our own real time portfolio management going on within Stable too. That’s what’s so attractive to underwriters and why many of them have offered to invest in us as well as support us with risk capital.”

People naturally tend to think about topping up their insurance only after the earthquake hits, or after the house burns down. Does the agricultural sector need a shock to the system before farmers want to buy?

(RC) That’s a clever question. While we’re educating farmers, there is a risk that some only think about ‘hedging’ when prices may have already fallen. And of course you’re right, demand will change throughout the cycle. Let’s say that demand in the UK is currently dominated by wheat and dairy. We can manage that, because we are working with lots of other products in lots of other countries.

Farmers don’t just face price risk, they also face income risk as well that ultimately hits their bottom line.

(RC)” Absolutely, and covering both price and cost risk is something I’ve always wanted to do. Next year we will also help the farmer insure against an unexpected increase in fertiliser prices or feed for example.
We’re also getting lots of approaches from food manufacturers [looking to hedge their input costs]. And many of them were like we don’t understand it [derivatives] either. We built Stable focused on making it simple for farmers, but along the way I realised that there are a lot more people in the supply chain that are confused about derivatives, than get it.”

Agricultural productivity is woeful. Many of the Baby Boomers that still own and operate farms both in the US and in Europe are reluctant to invest in new technology that would help them increase their yields and return on investment.

(RC) I believe this is totally solvable if inventors and developers rigorously focus on the economic case for their new invention or application. Building software is easy- solving a real problem is hard. If you can make it practical and show that it will clearly increase profits within a reasonable time period, then farmers will respond. “I meet many really smart developers who are developing clever tech that stacks up in the lab, but not in a field. The best thing they could do is close the laptop and pull on a pair of wellies! If the farmer doesn’t know where his revenue will be in 12 months-time, that’s a big barrier to investing in new technology to increase productivity.”

So what is the future for the agricultural sector? My son is 7 years old; would you recommend a career in farming for him in 10-15 years-time?

(RC) “I genuinely think it’s a very exciting sector to be a part of. All of this new technology is coming through and farming will be transformed in the next 5-10 years. It’s a really exciting place to be in right now. What’s really noticeable is that the sector is attracting some really bright people that see the potential to make a real impact in surely the most important industry in the world.”

Resources

Stable www.stableprice.com

Risk management schemes in EU agriculture: Dealing with risk and volatility https://ec.europa.eu/agriculture/sites/agriculture/files/markets-and-prices/market-briefs/pdf/12_en.pdf

Unlock commodity market insight now and subscribe to our email updates or

Pigs might fly: A look at the market for lean hogs

Sometimes its worth looking at markets off the beaten track of most commodity analysts, ones not beholden to movements in the US dollar or other broad macro economic movements. This week I’m looking at the market for lean hogs in the US, a market where cycles (a feature of all markets) is particularly pronounced.

Chinese pig herd shrank in September by the largest amount since early 2016. Beijing has shuttered thousands of small hold farms across the nation in a drive to impose tough new pollution standards by December, boosting domestic hog prices. In the past high pig prices in China has spurred strong imports of hogs from the US to meet expected demand.

Meanwhile, in the US higher feed prices (corn and soybean meal prices jumped last week ) will raise costs for US farmers increasing the cost of producing each hog and potentially restricting supply. The hog/corn ratio is currently 17.7 which points to stable pig output over the next 12-18 months, however an increase in the price of corn will, all other things being equal lead to a reduction in pig output going forward.

Hogs prices typically show strong seasonality*, with prices peaking in April / May and then bottoming out at the end of August. The opening up of two new major processing plants in September has put a floor under the market as they fight for feedstock and should result in the strong seasonal trend continuing through the fourth quarter.

Over the longer term, hog prices typically take about 3-4 years to move from peak to trough and a further 3-4 years to go full cycle. Hog prices just completed a full cycle and hit the trough exactly 12 months ago in mid-October 2016. All of which points to the start of a major turning point for lean hog futures.

* Lean hog prices show strong seasonality due to biological factors and the impact that the weather has on both supply and consumer demand. Since most pork in the US is sold fresh you cannot carry product from one time period to the next which tends to accentuate the seasonal tendencies.

Related article: Livestock prices: The top 10 most important drivers

Unlock commodity market insight now and subscribe to our email updates or