Its all about incentives

Earlier in May I had the opportunity to be interviewed by former hedge fund manager Mike Alkin. One of the things we discussed was the power of incentives, how they can crop up in unexpected places and the reason investors in commodities and indeed all financial markets need to be wary of them, especially when considering advice on where markets will go in the future. You can listen to me talking to Mike from 58.44. In this article I expand upon my interview with Mike.

“Never, ever, think about something else when you should be thinking about the power of incentives.”

– Charlie Munger

You will often see reports on financial TV, social media and the press pick up on a story that one or another famous investor or investment bank is bullish on this or bearish on that. Should you follow them blindly and invest? No, absolutely not. You don’t know what level of risk they are taking to enter that position. Have they hedged it and how? Do they even have any “skin-in-the-game” anyway and are following up their forecast with a position in the market? The best investors should also be able to change their minds if the evidence no longer supports their original hypothesis, and the same can be said for institutions that publish price forecasts. However, if the bank changes its mind will it get picked up by the press, or even get published? It might do, but it might not.

Think about why the financial media publish forecasts next time you see them on TV or on the internet. Are they to help inform those not fortunate enough to be able to afford buying their forecasts off the shelf? Are they meant as “entertainment”, as a way of gathering publicity to sell other “services”? Or are they setting up the sucker to invest in a particular commodity market, just as the forecaster has pulled out of the market or, even worse, is on the other side of the trade?

Various consultancies also operate in different commodity markets. Often they might be involved in collecting prices for opaque commodity markets, while using their experience and understanding of the underlying physical market to provide forecasts. Others might operate outside of commodity markets altogether, relying on their macroeconomic skills and models to provide forecasts.

At first glance these commodity forecasters might be seen as being the most independent from the actual market, since they have much less incentive to tailor the forecast to suit their client base or a position that they hold. That isn’t necessarily true however and ultimately depends on which of its client (buyers or sellers) have the most money to spend. Mining companies and other commodity producers often commission consultancies to provide ‘independent’ commodity forecasts to go into their prospectuses. On the other hand if buyers hold the purse strings in a particular commodity market then, incentives being what they are forecasts and analysis may be well be tailored to what that client wants.

Miners, commodity trading houses and others often release their own forecasts of where they expect one or a range of commodity prices to be in the near or long-term future. In the case of a mining company, these forecasts might be released around the same time as annual reports detailing the company’s activities are published, or when they are trying to raise funding for new investments. These forecasts can be said to have “skin-in-the-game”, with many commodity investors hanging on every word for clues as to how underlying physical demand and supply is likely to evolve. On the flip side, it’s difficult to argue that they are an unbiased prediction of commodity prices.

Forecasters are exposed to other incentives that may influence their behaviour. Researchers at the European University Viadrina Frankfurt (EUVF) analysed over 20,000 forecasts of nine different metal prices over different forecasting horizons during the fifteen years between 1995 and 2011. Instead of finding the institutional inertia and forecasting herding that we might expect, they found strong evidence of “anti-herding”.

So why might some forecasters want to stray from the herd? According to the EUVF paper it all comes down to incentives; and the incentive to herd or stay away depend upon the mix of clients, both existing and prospective. Think about who buys commodity forecasts. There are two groups of buyers. The first are those that buy forecasts regularly, perhaps as part of a subscription to a company’s analysis or for free as clients of an investment bank. Examples of frequent buyers of commodity forecasts might include an oil company or a manufacturing company that regularly buys a certain small range of commodities. Given they are long-time consumers, they may have based their decision on how accurate a forecaster was over several forecasting periods.

In contrast to the regular buyer, there are also onetime or irregular consumers of commodity price predictions. This second group of buyer is more likely to be swayed by the commodity forecaster that was most accurate in the past year or so, or has been the most vocal about his or her success. This is rational from the irregular buyer’s point of view in that perhaps movements in the price of copper or another commodity only have a minor impact on their business, or maybe they only need to buy or take a view on a commodity infrequently. Either way, the cost/benefit of monitoring whether the commodity forecast they are buying has been accurate in the longer term is much higher than in the first group of buyers.

If the second group of buyers dominates (the infrequent consumer), forecasters have a strong incentive to differentiate their forecasts from the predictions of others by making extreme (or non-consensus) predictions. Even though an extreme forecast may have a small probability of being accurate, the expected payoff of such a forecast can be high, since the number of other pundits making the same extreme prediction is likely to be small. Should they be successful in their prediction, then the forecaster can capture the attention and the wallets of the infrequent consumer of forecasts.

In contrast, if a forecaster publishes a less extreme forecast, one close to the consensus forecast, then by definition there is a high probability that other forecasters will make similar forecasts. If this is the case then even if a forecaster’s price prediction is spot on, then the impact on his income and reputation will be minimal. The infrequent buyer will ask, “why pay for a forecast from an average forecaster?”

Herding behaviour isn’t specific to explaining how the commodity forecasting firm appears to the outside world, it can also affect internal incentives. Career concerns can also play a part too. Just as at the level of the firm (whether a bank, consultancy or something else), you might think there is the temptation for an analyst to produce a bold prediction. If the analyst makes an “outlier” forecast that turns out to be spot on, this is likely to capture a lot of attention in the financial media, raising the prospect of the analyst being recruited by a rival firm touting a bigger salary and an even bigger bonus. However, set against this is the risk of being fired (or at least having a few rungs taken from under the career progression of a young analyst) for a bad call.

To examine what the age and experience of the forecaster has on the degree of herding, research published in the RAND Journal of Economics examined over 8,000 forecasts by equity analysts between 1983 and 1996. Equity analysts should produce reliable forecasts of future earnings of the companies that they monitor, which are then used to produce recommendations on what their clients should buy. Equity analysts face their own quandary, having to balance the interests of the buy-side (ie, their clients who prefer accurate forecasts) and those on the sell-side (other parts of the same bank they work for that might value trading commissions and large initial public offerings more than the accuracy of their analysts forecasts). Note that commodity analysts may face their own conflicting internal objectives too. From trading commissions on a commodity-related exchange traded fund, to a bank’s own proprietary trading on commodities and on to gaining profitable consulting business from a highly valued client. There is more than one incentive.

What the researchers found is that younger analysts tend to herd more than their more experienced colleagues do. Less experienced analysts, meanwhile, are more heavily punished for getting their forecasts wrong and so they have every incentive to stick with the herd. In contrast, older analysts, who have presumably built up their reputations, face less risk of termination. The researchers also found that, contrary to expectations, making bold and accurate predictions does not significantly improve a young analyst’s career prospects.

The final type of herding is known as investigative herding. Investigative herding arises when investors trade similarly by reacting to the arrival of a commonly observed information signal. Analysts have an incentive to investigate a piece of information or a market that he knows other analysts may also investigate and trade in. From the point of view of getting a return on the forecast, there is no incentive to build a position in a particular market if other investors won’t join on the same side and push the price in the direction of the forecast. Illiquid markets tend to be less well served precisely because it is not worthwhile for banks and other financial institutions to trade them. It follows that the reputational risk of making forecasts about illiquid and volatile commodity markets is much higher.

Incentives, they’re everywhere if you know where to look. Ignore them at your own risk.


Unlock commodity market insight now and subscribe to our email updates or

Positioning analysis in commodity markets: An interview with Mark Keenan

I recently had the opportunity to talk with Mark Keenan. Mark is Managing Director, Global Commodities Strategist & Head of Research Asia-Pacific at Société Générale. He is also the author of the book, “Positioning Analysis in Commodity Markets – Bridging Fundamental & Technical Analysis”.

In this interview you will learn why it’s important to be aware of the objectives of all market participants in commodity markets and why using methods like positioning analysis offer very good insights into whose doing what, and what happens when they do too much of it and how much positioning then relates to other factors like price, curve structure and fundamentals.

What first got you started in commodity markets?

I think like perhaps many of us I didn’t study what I ended up doing, much to some peoples disappointment, but it’s turned out ok. I studied chemistry and biochemistry so I was also drawn into the world commodities because I felt I had some edge in understanding things like the chemistry of oil and how its refined, metals, and how they are produced, including the various smelting and refining processes, and by extension in agriculture the genetic modification of crops. I liked all of that, understood it and I figured commodities were real. I studied science and that was real, and so I thought that if I’m going to get into finance, this is a suitable place with some degree of overlap.

The thing that really got me into modelling and trading it was that I was quite ‘quanty’ at university where even the chemistry that I did, was quantitatively driven. I worked on the human genome project which is not so different from what I do today, which is looking for patterns and sequences in vast amounts of data. Specifically, at the time when I started out, it was the birth of computerised trading in so far as all the CTA models and trend following algorithms were starting to proliferate and develop. This was much in alignment with the growth in software that became available at the time to test and build strategies such as TradeStation and Metastock. I spent my early days endlessly building and programming algorithms and of course commodities were one the main markets that leant themselves well to that approach.

The “real” nature of commodities and a background in modelling were the two main things gave me my background in commodities and seeded my interest in the asset class.

What gets you excited about talking about commodities?

I think it’s important to make a distinction between talking about markets and talking about commodities. The thing that is so fascinating for me about commodities is that they are so very different from the other financial assets. You have such a vast array of objectives among the many different market participants which adds huge insight into behaviour. Very simply, if you look at the futures markets, which is the main point of access to commodity markets for most people, you have consumers and producers which are essentially “hoping” to lose money with their derivative hedging positions and make money in the physical world; you’ve got speculators hoping to make money with directional bets, but in reality, many are playing relative value positions, and collectively all of these groups are trading across a forward curve, so the point or location in time at which they are doing it, also varies considerably. So that’s hugely interesting, and distinct from the relatively uniform objective of participants in the equity market for example, space where pretty much everyone is seeking to make money out of their position.

I think also that uniquely in the commodities space, there is often a mismatch between analysis and research and the actual price response despite often considerable consensus in a view that make the asset class tradeable. By that I mean that you often come across situations, where you’ll see that many people are bullish sugar for example, and because of transport dynamics, including port and vessel logistics and many physical characteristics of the market; everyone may think the market is going to go up, but they may not be able to effect that view. Simple they might not be able to buy all the sugar that they need now, because they just don’t have the budget or the credit lines or the vessels to do it. By doing research and understanding what the underlying fundamentals are, because of these structural hurdles, there is often a slower response in terms of how the price catches up with the view. This allows positions to be taken from an investment, trading or speculative perspective that have time to make money.

As many of these logistical constraints force shifts in fundamentals to persist for some period of time, prices often become very serially correlated. Interesting mismatches between research and analysis and the eventual price response occur that are often monetizable due to these relationships not being contemporaneous (like in many other asset classes). I like that disconnect, that’s what you can get your teeth into, and that’s what can make money.

Even for speculators, I would say that 70% are CTA driven in the commodities space which generally means they have a rules based strategy or they are technically driven and are hence often constrained in how flexible and responsive to a view they can be. For other types of investors, whilst they may think a commodity may go to price x they are often subject to enormous restrictions in terms of their trading and investment mandates and can’t really effect the view either. Lastly on and almost on a daily basis, I come across people that have a view on a particular commodity, whether its natural gas, gold or crude oil and they don’t really understand how to capture that view. They are simply not familiar with the access instruments, or they are not comfortable with the instrument like futures or options.

Collectively many of these properties and dynamics, combined with the innate diversity across the complex make commodities fascinating assets to trade in a financial framework. Their sensitivity to their own endogenous idiosyncratic price drivers, but also to outside macroeconomic variables combined with how well they lend themselves to data analytics make them fascinating to talk about and to study. They are always changing and there are always new ways of looking and thinking about at things, fundamentally, technically and for a sentiment/behavioural and positioning perspective – that’s what I like.

How should traders and investors use that blend of approaches – fundamental, technical and positioning?

I get that all of the time when I’m talking to people who are just entering the markets. Should I be a technical or a fundamental trader? And what I hope to try and do is to break down many of the preconceived barriers or notions about what types of analysis works and to not think about research or trading in silos. I don’t think that makes sense anymore, or whether it ever did. A blend of a variety of different analytics is essential, especially with the growth in technology, and the fact that everyone with a website can now plot reasonably advanced technical analysis for all commodities, as distinct from 20 years ago when they had the point and figure charting on the wall of the NYMEX exchange, and price data was very hard to obtain. Now, with it being much more accessible, I think a more balanced exposure to different ways of analysing the market is very sensible, with the challenge very simply lying in knowing when you need to look at fundamentals, when technical analysis is important and when the slightly more esoteric things like sentiment and behavioural patterns and positioning can help. This is what makes the difference between a good trader and an excellent trader.

You say that the individual can produce relatively sophisticated analysis but isn’t the playing field still weighted against them when they are up against commodity trading funds?

That’s an interesting question. I think most certainly the physical market participants including the trade houses have a phenomenal window into the actual commodity market and in many instances influences over the movement and storage of commodities all over the world. This affords them deep insight into how commodity prices are going to behave. They do however tend to be less familiar with the investment world, which has a significant interest in commodity markets and is growing by the day, with how prices are behaving. They have very little understanding of how investors behave and what they want out of commodity markets. I’m constantly sitting down with trade houses, that might be bearish soybeans for example, and I say I’ve been talking to a pension fund that’s just increased their commodity exposure, which means they will be buying soybeans. Often they can’t reconcile this and therefore don’t understand why they still buy them. They’re not familiar with commodity indices, diversified exposure and hedging against inflation and all the other objectives.

It’s important to be aware of the objectives of all market participants, as the market continues to grow and become more diversified – especially now with computerised trading strategies and algorithms. A lot of the physical world is very nervous about this; they don’t understand what this might mean for prices.

In short, with so many different market participants, and a fascinating blend of different strategies, using methods like positioning analysis offer very good insights into whose doing what, and what happens when they do too much of it and how much positioning then relates to other factors like price, curve structure and fundamentals. This can help all market participants enormously – especially individuals.

How should someone think about a market where money managers are holding a large net short position? That it could unwind and so see prices rise sharply, or that money managers have typically done well in the past and so prices are likely to stay lower for longer?

There are a few factors to take into account; firstly it is very important to have some view as to whether the market is range bound or whether its trending – be that up or down – since this can alter the interpretation of positioning quite significantly.

Secondly, one of the things I have found interesting, is that money managers and commodity speculators since 2008 have had a very difficult time making money. If you track hedge fund indices based on the performance of commodity hedge funds, they have been in near linear descent. Anecdotally, we also continuously hear reports of hedge funds winding down. The question therefore of what to do when a manager has large position lies in whether to copy them or to bet against them.

One of the things in the book that was addressed is how to measure where money managers are making money. Understanding this is very important as it can help shape behaviour. For example, hearing that fund managers are very long natural gas for example – a market that they’re generally very profitable in, could suggest that not going against this trend would be sensible.

Perhaps the most important thing that I think is critical is that positioning analysis cuts traders up into various buckets and then they have to behave accordingly to what bucket they are in. For example in the money manager bucket they have to close out their positions, they can’t go to delivery, they can’t play in the physical market because if they could they would be in a different category. So when you see a very large short position that means very simply that they are going to have to close that position out. And if you see that in the context of a very low price suggesting that those money managers have made some money building up that short position and prices have fallen, that can be a very interesting buying opportunity for consumers as there is a high risk that those prices move higher as a function of short covering. And likewise on the other side when you have a very large long position and prices are very high, that might be a very good time for a refiner to sell some of their products. Simply using the position of money managers, whether they are right or wrong in their direction can really help you time your trading and hedging more effectively. We’ve found that many of the clients that we’ve spoken to and showed them these very extreme positioning its been very helpful.

What was your objective behind writing the book?

My objective very simply was to define positioning analysis as an area of research that is a very powerful framework for understanding how commodity prices behave. And I think that positioning data is probably the most underutilised set of data in the commodity markets. Over the years through building models I’ve realised that it is incredibly powerful so the objective behind this book was to set out numerous strategies and models that show different ways of thinking about how positioning data firstly bridges fundamental and technical analysis and how changes in positioning patterns inter-plays with price, curve structure, inventories, cancelled warrants in the metals sector, seasonal factors and many others. It’s fascinating as you’re identifying how people behave in the context of a wider variables and that really helps trading.

People have said to me that one of the biggest hurdles to using positioning data is that it is lagged. But my response to that is that every bit of data is lagged. The fascinating thing to me is when I ask them what you mean by the word lagged. They seem to infer that someone has the data before them, that in some ways that it has been corrupted in some way. When you look at inventory that people often attach even more importance to that is lagged and that is data that is most definitely corrupted because the people that control the inventory and have built storage facilities they know pretty much before the market participants. If you take energy as an example when the Dept. of Energy release their inventory numbers all the oil market has a very good idea of what’s going to happen, yet people don’t talk about that as being lagged. Whereas with positioning data every single person in the world, except the US government gets it at the same time. When you look at it like that that allays some of those concerns.

So my objective with the book was to break this data apart and blend it with all these inputs, that we often look into isolation, and try and tie them together into very simple models. The look to see what happens when these variables reach extremes and when they move in trends. What does that mean for prices, curve structure and behaviour patterns. Certainly over the years I’ve found these relationships to be extremely powerful.

There’s a chart in your book that shows Bloomberg search terms for positioning, and while its clearly become more popular in recent years it’s still not mainstream.

I was trying to explain positioning to people that are not in the market and on a very basic level its like gossip. Everyone is interested in that on some level. If you go to a technical analysis in commodities and you say to him that copper inventories are at a ten year low, he or she is not going to really care about that information. If you go to a fundamental analyst and you explain that the MACD lines have crossed over last week they are not going to know what that means potentially. But if you go to both of them and you say guess what all the consumers did last week? They are not going to say we’re not interested. That’s useful information. How you cut it and deliver that information is crucial. As you say there is a big gap between technical and positioning analysis and positioning analysis fills that well.

Any plans for a future book?

People have asked me to extend the book and do a more advanced version with all the latest data – a sort of advanced positioning analysis that might build on this.

One of the charts in the book is based on news flow. One of the areas we work a lot here at SocGen is building news flow indices to do all sorts of things, to assess geopolitical risk, the weather, trends in fundamentals and price and so on. I think that news flow analytics is doable now because of the computer power available now and the integrity of news databases. Perhaps my second book will revolve around news flow analysis as a new type of momentum signal. It’s quite fascinating when you look at the correlation between a basic technical trend following model for example – one where the signals are generated not by changes in price, but by changes in news flow, where people talk about copper going up or down for example. When you look at that dynamic, it tends to give you a very different return profile.

Thanks for your time today Mark. Where can people find or follow you?

You can follow me on Twitter @markjskeenan

Unlock commodity market insight now and subscribe to our email updates or

Soybean prices: The top 10 most important drivers

1) Weather

The weather has a significant impact on crop yields and thus overall agricultural production. The wrong type of weather at the wrong time in the planting cycle, even if not prolonged or extreme, can also adversely affect the production of soybeans. The weather can also affect the shipment and logistics of transporting crops to market.

Heavy rain in the north and west-central regions of Brazil (where most of the worlds crop is grown) makes working conditions in the field more difficult and reduces the planting pace, in turn hurting the soybean output. Meanwhile, adverse weather conditions in the south of the country (where most of the crop is shipped from) could affect the supply of soybeans out of Brazil while delays could also increase wastage, further reducing supply.

2) Protein demand

The importance of China and other markets in Asia as a market for soybeans has been driven by an explosion in demand for meat as consumers switch from a diet dominated by rice to one where pork, poultry and beef play an important part. Soybean meal (a product of soybeans) is the most important protein source used to feed animals and represents about two-thirds of the total world output of protein feed-stuffs.

3) Chinese demand

China imports around 60% of global soybean imports, around 90% of its requirements. Chinese production of meat surged 250% between 1986 to 2012. However, China is unable to produce enough animal feed itself, hence the need to import soybeans from the United States and Brazil.

4) The US Dollar

Like most internationally traded commodities soybeans are priced in US dollars. At its most basic a decrease in the value of the US dollar relative to a commodity buyer’s currency means that the purchaser will need to spend less of their own currency to buy a given amount of the commodity. As the commodity becomes less expensive demand for the commodity rises, resulting in an increase in the price and vice versa.

A weaker dollar can also act as a disincentive to producers to increase output. For example, a depreciation of the US dollar against the Brazilian real can reduce profit margins for a soybean farmer in Brazil. All of the farmer’s revenues will be received in US dollars, which will now buy less real, but some proportion of the costs will be denominated in real and will remain constant (at least in the short term). Therefore, the prospect of a lower profit margin acts as an incentive to decrease the supply of soybeans.

5) Tariffs

Given soybeans position as one of the most important sources of food governments may use this as a key element of trade policy with other countries. In 2018 for example China announced that it would increase tariffs on imports of soybeans from the US. The impact of such a tariff would be to increase the price of soybeans in China, but for the price to fall in the US.

6) The price of other crops

Corn and soybean compete on multiple grounds. They compete in the cooking oil industry. They compete in the animal feed industry. They also compete in the biofuel industry. Therefore, the production of one does affect the other. In general, if the production of corn falls, soybean prices are expected to rise.

7) Stock levels

Stocks (otherwise known as inventories) act as form of buffer for both producers and consumers of soybeans. Typically, falling stock levels occur if demand increases faster than supply, resulting in higher soybean prices. Falling stock levels may, however, make the soybean market more vulnerable to an unanticipated disruption to supply or a sudden increase in demand.

8) Speculation

Hedge funds operating in agricultural markets including soybeans may result in prices moving faster than they otherwise would in the event that there is deficit or surplus in the market. Speculators play a vital role then in signalling to farmers how much they should be planting while also allowing the transfer of risk.

9) Energy costs

Higher energy costs imply higher costs of production for soybeans and higher costs of transporting soybeans to market. Energy makes up a significant part of operating costs for most crops. This is especially true when considering indirect energy expenditure on fertiliser because the production of fertiliser is extremely energy intensive, requiring large amounts of natural gas.

10) Speculation by funds

Over the short term at least the positions held by large commodity funds can have a significant impact on soybean prices. If positions in the soybean futures market become extremely overbought or oversold then, as in other commodity markets news that counters this view can result in a sharp upswing in price volatility in the market.

Previous episodes

Gold prices: The top 10 most important drivers

Silver prices: The top 10 most important drivers

Natural gas prices: The top 10 most important drivers

Copper prices: The top 10 most important drivers

Livestock prices: The top 10 most important drivers

Sugar prices: The top 10 most important drivers

Cocoa prices: The top 10 most important drivers

Palladium prices: The top 10 most important drivers

Unlock commodity market insight now and subscribe to our email updates or