Demand and supply: Rewriting history

“No other industry begins to offer the data problems that are presented by petroleum,” – John Blair

The production of basic statistics and forecasts about oil reserves, production, consumption and stocks ought to be a matter of routine. You just stick a gauge at the end of a pipe and measure the amount of liquid flowing through (in the case of oil) right? Unfortunately, it’s not that simple, and the problem isn’t limited to just oil, but all commodities.

Unlock commodity market insight now and subscribe to our email updates, follow on Facebook or

Doubts about the reliability of energy statistics were a major part of the “energy crisis” that erupted during the 1970s. As late as 1968, the United States had an estimated 4 million barrels per day of spare crude production capacity and thousands of wells across Texas and Louisiana were being operated for fewer than 10 days per month. But just four years later spare capacity had suddenly dropped to zero, every well was at maximum production, domestic output was falling, and politicians began to speak of an energy crisis.

The OPEC oil embargo, announced in October 1973, intensified the sense that something had gone badly awry, leaving the USA unprepared. Politicians and the media blamed a conspiracy between domestic producers and OPEC for engineering the crisis to drive up prices and profits.

Congress held hearings amid a sense that the statistics and forecasts prepared by oil and gas producers and the U.S. Department of the Interior had been either inaccurate or deliberately manipulated. One outcome of the crisis was the creation of a new U.S. Department of Energy and within it a new Energy Information Administration (EIA) in 1977 to produce more accurate and independent data. Another was the creation of the International Energy Agency (IEA) in 1974 to gather better statistics and bring greater transparency to the international energy markets.

Prior to the energy crisis, most data and forecasts were confidential and under the control of oil and gas producers themselves. After the energy crisis, data collection and forecasting would be led by impartial civil servants at national and international levels.

Improvements in data collection and forecasting in the United States, led by the EIA, have largely quelled controversy about domestic US oil production, consumption and stocks. But that doesn’t mean they are free from error or revision. Indeed, information on international markets remains much less comprehensive and accurate, mostly as a result of data collection problems in emerging markets and the deliberate secrecy of the major oil producing countries, particularly those who are members of OPEC.

According to a recent study by the Wall Street Journal, annual estimates of global crude demand by the IEA have been revised up for the past seven years (up until 2016) by an average of 880,000 barrels per day. And there is little evidence that demand estimates from other institutions are any more accurate. The EIA have also underestimated consumption over the past seven years, with the annual figures being revised up by an average of 2.3 million barrels a day.

Revisions to oil supply estimates are typically much smaller than for demand and are often about correcting overestimates for crude production. The IEA’s supply data has been revised down 60,000 barrels a day on average over the last seven years, according to the Journal analysis. That means the oversupply usually ends up being smaller than initially thought.

Demand is much harder to estimate than supply. Unlike supply which can be estimated from the pre-announced expansion plans of a relatively small number of companies, estimating demand involves billions of consumers worldwide and many millions of companies of all shapes and sizes.

The history of discrepancies underscores how commodity markets often trade based on, and pundits provide price forecasts using incomplete and often significantly revised data. And remember, the oil market is by far the largest, most liquid, most important commodity market in the world. If such big revisions are made in the oil market, then imagine how difficult it becomes to estimate demand and supply and then forecast prices in much smaller markets like lead, live cattle or lithium.

Buy the book Commodities: 50 Things You Really Need To Know from Amazon (US and UK), iBooks, Barnes and Noble, Google and Kobo. If you like the book please leave a review on Amazon. Reviews really do make a difference. Thanks.

Where do we stand in the commodity cycle?

Many investors in commodities could be forgiven for thinking that the good times are back. Since the start of 2016 one measure of commodity prices is up over 41%. The best performing commodity, natural gas is up over 70% in the past twelve months while both crude oil and zinc are up around 50%.

Investors and other with an interest in the outlook for commodity markets might assume that because some commodities have gone up sharply then others are sure to follow or that the recent increase in commodity prices heralds more extensive, longer term gains. Indeed, net positioning among investors in commodity futures contracts is at record levels, reflecting optimisim of further gains. The question is more than just academic. Investors in particular might be emboldened by recent gains and think that this is just the start of a whole new super cycle.

Unlock commodity market insight now and subscribe to our email updates, follow on Facebook or

But rather than try and know the unknowable (i.e. the future) it far better to try and understand where we are now in the cycle. In the world of investing nothing is as dependable as cycles. Commodity price cycles rise as demand overwhelms the pace of supply to respond before eventually supply overwhelms demand before the cycle eventually returns to square one, and it begins again.

Research examining centuries of commodity price data has tended to sketch a pattern of 15–20 year super-cycles (a period of rising prices), followed by a slide in prices over the following 10–15 years when excess investment leads to a flood of supply.

The most recent super-cycle in commodity prices had its origins in 1998, when prices approached their lowest for 20 years (equal to depression levels, when adjusted for inflation). This was followed by the emergence of China as an industrial force, which drove commodity prices up in the first decade of the 21st century. Most industrial and agricultural commodity prices peaked in 2011 with oil peaking in mid-2014.

Stating the obvious here but that would mean that the down cycle in commodities has been less than half of previous commodity cycles. Is that really realistic? Has all the excess baggage (debt, overcapacity etc.) that was built up in the boom times really been rung out of the system?

A cursory look at any long term chart showing nominal commodity prices shows that even within long periods of rising or falling prices there are still lengthy periods when prices run counter to the longer-term trend. In addition, not all commodities will necessarily behave in the same way during the period of a commodity cycle. We have highlighted before that commodity prices are prone to sharp price spikes or drops even within much longer term cycles.

Related article: Copper and oil prices: Lower for a bit longer?

And that begs the question, if this is the start of a new super-cycle then what are the catalysts? And more importantly are they strong enough to drive commodity prices back to and beyond the highs achieved in 2011?

The prospect of a weaker US dollar and rising inflation expectations are perhaps the main two macro factors that have driven investors towards commodities over the past year. Coupled with a sense that prices for many commodities had hit a bottom early in 2016 funds have moved back towards the sector. Nearer term a rapid expansion in credit in China as well as curbs on production at domestic coal plants has increased demand for seaborne coal, in turn boosting prices of iron ore and many other metals. Producers have also taken direct steps to influence the supply-side too from Glencore’s decision to shut-in significant amounts of zinc output to OPEC’s agreement to cut output. But arguably there isn’t yet at least a powerful enough driver of commodity demand (or persistent curtailment to supply) on the horizon that could drive gains longer term.

Most commodities are arguably then still in the normalisation part of the commodity cycle. This is where oversupply and low prices incentivise production cuts and encourage incremental demand growth. Once through this there is the rebalancing phase as falling supply and rising demand lift prices, eventually returning the supply-demand balance to equilibrium.

History suggests that this rebalancing phase tends to be associated with the sharpest increase in prices. This is probably the phase that industrial commodities such as zinc, lead and cotton and more recently copper and oil have been going through. Agricultural commodities such as corn, soybeans and wheat could be next.

Once commodities move through the rebalancing phase, the trajectory typically becomes less certain due to the length of time and the many variables involved. Analysis of historical cycles by Cohen & Steers estimated that commodities move from rebalancing to tightening approximately 70% of the time.

What is the inference from the rebound in commodity prices? Investment banks, other pundits and the media are becoming increasingly confident in their attitudes to commodities. Although the peak might not be here just yet, it always pay’s to look to the opposing side when one side of the boat becomes too crowded. Goldman Sachs recently likened the credit driven rally in commodity prices to the 2008 boom in commodity prices. There is no sense, yet at least of euphoria in the market where investors are getting ahead of themselves.

The size of the net long position will require prices to increase further or otherwise risk investors pulling funds out of commodities. The possibility of a stronger US dollar coupled with rate hikes in the US could put a lid on further gains in commodity prices. A sharp rise in US shale crude production combined with a return to historical levels of OPEC compliance could finally break out the narrowing in crude price levels to the downside. China meanwhile may well slow during the rest of 2017 putting pressure on industrial metals in particular.

There is a possibility that this commodity super-cycle and those of the future may become shorter and shallower. More data and importantly more accurate data on the stock of commodities in use at any one time, the expansion of ‘manufacturing like’ processes in US shale oil and gas to other commodities and more liquid and sophisticated hedging tools may enable both producers and consumers to more accurately balance commodity supply and demand.

That could be the future. With oil being one of the primary driver of cost across many other commodities a stable equilibrium around $50 per barrel could prompt less volatility in other commodities. The more likely outcome though is the way it has always been, that oil and other commodities are never in equilibrium, always moving further away and towards it but never stable.

For now though be cautious about extrapolating gains in industrial commodities out too far. There are likely to be further disappointments ahead before fundamentals justify higher prices.

Buy the book Commodities: 50 Things You Really Need To Know from Amazon (US and UK), iBooks, Barnes and Noble, Google and Kobo. If you like the book please leave a review on Amazon. Reviews really do make a difference. Thanks.

Curve appeal: Why investors looking to profit from higher commodity prices need to look to the future

The rebound in many commodity prices since the start of 2016 has reignited investor interest in moving part of their portfolio into commodities. Generally, long only investors (betting the price will go higher) will use an Exchange Traded Fund (ETF) or even through using other instruments like spread betting and Contracts for Difference (CfDs).

What they all have in common is that they all use commodity futures contracts, either as the route to which the investment is made or as a derivative of the underlying commodity futures contract.

Unlock commodity market insight now and subscribe to our email updates, follow on Facebook or

However, just looking at a chart of prices over the past 12 months or so does not show you the full cost of gaining an exposure to one or another commodity. You need to look at the shape of the curve. Anyone not doing so might be in for a shock.

The curve we are talking about is known as the futures curve and is represented by a series of prices for contract dates stretching a year ahead and beyond.

A futures curve is described as “in contango” when it is upward sloping and so prices in six months’ time are higher than the spot price. This is also known as a normal curve or a normal market. In general, traders are willing to pay a premium to avoid the costs associated with transporting, storing and insuring a commodity (known as the cost to carry); therefore, the furthest-out contracts are typically higher in price.

In contrast, when the shape of the futures price curve is downward sloping, the futures price of a commodity in say six months’ time is lower than the current spot price, and so the market is said to be in backwardation. This is also known as an inverted curve or an inverted market. If a futures curve moves towards backwardation (also described as a tightening in the futures curve), it is a good sign that the current underlying conditions in a commodity market are getting tighter – either via gradually improving demand or supply problems, or a combination of both. The opposite of which is so when the curve moves towards contango.

The shape of the futures curve has significant implications for commodity producers, consumers and investors. In the event that a commodity futures curve is in backwardation, producers might choose to run down stocks or look to increase output, taking advantage of the tight market and fearing that they could get a lower price in the future. In contrast, if the futures curve is in contango, a producer may want to put the commodity into storage and sell it at a higher price in the future.

Investors first hearing the term “contango” might guess that it’s a variation on an Argentinian dance, but the word actually refers to a condition in the futures markets that is especially costly for long only (i.e. betting that prices will go up) investors in commodity futures markets.

The difference between spot and futures prices for a commodity is known as the basis. The price of a futures contract – whether it is above or below the spot price – will converge to the spot price as the expiration date on the contract approaches. This process is called convergence. For someone holding a futures contract where the market is in backwardation, the value of their contract will rise to meet the spot price, enabling them to achieve what is known as a positive roll yield, ie, a bit of income from selling one futures contract and buying another.

As of today, of the major commodity futures contracts listed on the US commodity futures exchanges only two are currently in backwardation – live cattle and orange juice.

The opposite applies for a trader holding a futures contract where the market is in contango. This time a trader who has purchased a futures contract where the market is in contango will be facing a loss on the contract roll.

So for example if you think wheat prices are going to rise the nearest traded contract is March and trades at 449 cents per bushel, whereas the next contract in May trades at 463 cents per bushel. If when the March contract expires the prices are the same then the holder of the March contract will sell his or her contract and then purchase a new contract at 463 cents per bushel – resulting in a loss of 14 cents per bushel. Even if both the March and May contracts increase in value by 14 cents in the period to expiry you will have made zero profit on your investment, despite the higher spot price.

Buy the book Commodities: 50 Things You Really Need To Know from Amazon (US and UK), iBooks, Barnes and Noble, Google and Kobo. If you like the book please leave a review on Amazon. Reviews really do make a difference. Thanks.