To understand the effects of these forces on commodity trading’s traditional business model, it helps to examine the three main stages of the industry’s value chain: pretrade, trade, and posttrade. (See the exhibit.)
The first stage of the value chain consists of two elements: securitization and investment decision support.
Securitization. This can take one of two forms: securitization of assets or securitization of contracts (also known as origination).
The securitization of assets can involve both aggregation (for example, the pooling of solar panels to create a virtual power plant) and disaggregation (for instance, the division of financial ownership of a facility for exporting liquefied natural gas [LNG] or storing oil). Securitization is highly vulnerable to digital forces,
since both aggregation and disaggregation can be performed automatically on digital platforms: instead of each deal being negotiated by commercial departments and commodity traders, financial products can be grouped, regrouped, and traded instantly. This changes market dynamics by enabling more players, including small and nontraditional ones, to gain access to assets and to do so with very little capital, which can partly erode the advantage of owning and operating
The securitization of contracts involves creating standardized products from large-scale, nonstandard contractual agreements between two parties. Examples include agreements regarding the long-term off-take of LNG and structured investment products, including those based on energy consumption patterns. Many traders in less-developed commodity markets have created a business based on the securitization of contracts. Their business model will face growing pressure as commodity markets become increasingly developed and as more short-term markets emerge, offering greater liquidity, price transparency, and ability to hedge risk. That evolution is already evident in a number of markets, including European gas.
In addition to the risks for traders, however, there will also be opportunities. A trading firm could, for example, build an algorithm-based platform that gives price quotations on structured products in near real time and without the need for human traders. Such a platform would appeal to sellers, buyers, and intermediaries.
Investment Decision Support. This refers to the analytical activities associated with preparing a trading strategy. It includes price formation analysis, fundamental supply-demand modeling, and the quantification of market sentiment and customer preferences. These activities are changing drastically as new quantities and types of data allow market players to understand supply and demand patterns and market sentiment much more thoroughly.
Examples abound. Gaining access to data related to commodity pricing, for instance, no longer requires tapping the broker middleman; the data can be obtained via any number of channels. Raw data on the movement of global vessels that transport commodities is abundant and readily available. Fuel retailers can better understand demand patterns by installing microsensors at fuel pumps. Traders focused on crops can gain real-time data on crop maturation through smartphone videos recorded while driving by the fields.
Commodity traders also now have immediate access to new types of data that can help them understand future supply-demand dynamics. This includes data mined from social media and online news sources, which make data on market sentiment, weather conditions, accidents, and other relevant variables available in real time and often in advance of official news releases. Indeed, the principle challenge for traders now is not how to retrieve such information but rather how to organize and analyze it. An emerging set of businesses is focused on gathering data from multiple sources, structuring it, and feeding it directly to trading systems. RavenPack, for example, follows news from various sources and organizes it on the basis of impact and novelty; Quandl, a data-collection service, gathers and organizes data on prices and fundamentals, sparing traders the task of having to extract, transform, and load the data themselves.
The value of this growing abundance of information is being enhanced significantly by learning algorithms. The volume of data is simply too big for human-built analytical models to mine fully and with sufficient speed. Today’s learning systems, in contrast, can quickly crunch the numbers and readily identify patterns invisible to human traders. Machine learning, for example, can classify data, cluster principal components, and create decision-making frameworks based on data and algorithms independently, without need of human guidance.
The second stage of commodity trading’s value chain is divided into three parts: position generation, portfolio management, and execution.
Position Generation. This is the process of deciding which positions to take—what to buy and sell, with related decisions on quality, quantity, location, and timing. Digital technologies are transforming position generation by allowing automated decision making that is faster and more precise, and by driving down costs, including financing costs. Consider gas storage, for example. Optimizing a gas-storage position against forward prices is time-consuming and laborious work because a company must constantly readjust its position as the market moves. Indeed, large players often deploy teams of analysts to optimize their storage strategy, yet the companies still move relatively slowly and inefficiently relative to the market. Today’s digital technologies significantly improve both the speed and the economics of executing this strategy. At BCG, for example, we built a pilot optimization algorithm that allowed hourly, rather than daily, reoptimization and automatically readjusted itself in response to the market’s rapidly changing dynamics. The result was an increase in storage value of 8% to 10%. Real-time optimization could yield even greater benefits. Further, the automation of data and information feeds would allow companies to manage such a model with a relatively small support team.
We see similar opportunities in physical production. Oil refineries, for example, typically deploy an optimization model that helps calculate the best inputs of different grades of crude given existing demand and refinery configuration. Refineries have generally run this model weekly or daily; some players have run it a few times a day. Given the rapid evolution of digital technologies, we can easily imagine intelligent systems that permit real-time analysis of the technical and commercial variables, providing managers a minute-by-minute understanding of the optimum crude blend and product output. (Making this work, however, would require changing the refinery supply team’s operating model and ensuring very tight integration of data flows.) A current example of such capabilities in physical production management can be found in power generation, where data on supply and demand can be fed automatically to algorithms that execute trades to exploit any imbalances.
Portfolio Management. Managing a portfolio involves combining, prioritizing, and sanctioning trading opportunities on the basis of an assessment of the portfolio’s value and risk. Traditional portfolio management systems typically use streamlined return distribution assumptions to calculate outcomes—but this results in overly simplified conclusions. Integrated intelligent trading systems, in contrast, allow traders to perform much more realistic, path-dependent risk assessments as well as sophisticated scenario analyses. Performing such instant and comprehensive portfolio assessments requires massive computational capacity, which is now available from cloud providers and is thus available to all companies. What’s more, computing capacity itself is being traded today, adding another dimension to the trading-optimization possibilities and illustrating the general trend in markets toward commoditization.
Execution. This is the process of selling or buying commodities, whether physical or financial. Traditionally, trading companies’ transactions would be executed by brokers who worked in banks. Increasingly, however, machines are able to perform execution directly in the marketplace and to do so faster, more cheaply, and more accurately. They are also gaining the ability to do so in a more sophisticated manner that prevents other players from detecting the company’s intentions.
Consider, for example, an oil producer looking to hedge its risk exposure to 100,000 barrels of Brent crude oil. In the past, the company would ask its broker to execute the hedge; the broker would divide the hedge into blocks to smoothen the impact on the market and start to execute the hedge over the course of the trading day. Today, however, algorithm-based programs would sense this strategy and trade against it. As a result, any liquidity evident in the market at a given point in time might not be there when a company wants to execute the strategy—the bid-ask spread and the order book are very dynamic.
Intelligent trading systems can make a company’s execution less obvious and optimize the trades so that they have only a minimal effect on the market price. Such systems identify the best block size, which is often randomized, and the ideal timing for placing the hedge (for instance, when there is a high amount of liquidity in the market). These systems can also react to the behavior of other market players and do so dynamically and in near real time. A human trader could not possibly replicate this process, which explains why the technology is already in widespread use in such areas as foreign exchange and equities trading. Use of this technology is growing particularly rapidly in European power markets, where algorithms are taking over execution activities related to short-term trading. Traders that have begun to explore the possibilities here have been impressed by the results. To capitalize fully on the potential these systems afford, however, traders will need to make changes to their governance policies, which will present a large challenge to incumbents.
This is the tail end of the trading process, where the deal is settled, documented, and reported. Adoption of digital technologies is already widespread here, driven largely by trading companies’ desire to minimize costs by replacing manual work with straight-through processing. Some trading firms have also digitalized their back-office processes to improve the quality of their posttrade reports.
The use of digital technologies is also often a practical response to the increased reporting requirements driven by regulation, such as the Dodd-Frank Act, the European Market Infrastructure Regulation, and the Regulation on Wholesale Energy Market Integrity and Transparency. Regulation can also spawn opportunities. Traders that can carefully analyze trade repositories, for example, could achieve a better understanding of market dynamics.
In sum, digitalization is affecting all parts of the commodity-trading value chain. It is changing the way that information flows through it and, essentially, deconstructing the chain in the process. This progression will vary in speed and degree by market, however. How can managers determine where their market stands?