Applying Financial Machine Learning In The Real World - A Conversation With Dr. Ernest Chan
The following transcript is from the conversation between Steve Young of ENJINE and Dr. Ernest P. Chan.
Dr. Ernest P. Chan, is an expert in the application of statistical models and software for trading currencies, futures, and stocks. Dr. Chan holds a Bachelor of Science degree from University of Toronto (1988), a Master of Science (1991), and a Doctor of Philosophy (1994) degree in theoretical physics from Cornell University. He is the author of ‘Quantitative Trading: How to Build Your Own Algorithmic Trading Business,’ ‘Algorithmic Trading: Winning Strategies and Their Rationale,’ and ‘Machine Trading: Deploying Computer Algorithms to Conquer the Markets.’ Dr. Chan is the Managing Member of QTS Capital Management, LLC.
Steve Young: Dr. Chan, in your books you’ve covered a number of quantitative tools and techniques from tests for co-integration to Kalman filters, and, in your latest book, machine learning. What are some of your favourite quantitative methods and how do you apply them in finance?
Dr. Ernest Chan: Well, I think that in the past year or two, definitely machine learning is my favourite technique. I have been an expert in machine learning for a long time—since the 1990s. But, there has been a period when I have found this technique not to be particularly productive, if I apply it to trading. And, I was really plagued by the issue of data snooping, and over-fitting by it. But, it is only in the last couple of years that I have come to learn about new techniques in the field—especially in financial machine learning. Many of the ways to combat data snooping bias that I have found with the new understanding of how to apply machine learning to finance really drew my attention, and lead me to believe that it can be the most fruitful technique for adding to the quant finance arsenal.
Steve Young: That’s interesting. We can come back to the machine learning aspects later. But, as I recall from the first two books that you wrote, you focus primarily on linear models. But, if you have been aware of and had expertise in machine learning for a long time, is there a reason why you focused on linear models before, but you are leaning more towards machine learning models now?
Dr. Ernest Chan: Yes, I mean a linear model has the merit that it is simple and it is intuitive to understand. So, if a linear model fails, you can usually concoct a story of why it failed: “Oh, it’s because we are in a regime where volatility is too low”, or “because of the trade war”, or some story. And, you can say, “Okay, it’s not going to work in this environment because of the trade war”, let’s say, and, because of its simplicity, it is less in danger of being over-fitted. So that is both in the understandability of the model, and that, as well as the danger of over-fitting, caused me to favour the linear model in the past. But, on the flip side, because of its simplicity and its transparency, it is very easy to replicate. I never imagined that I have such unique insight that this linear model that I could use was so unique that no one else could come up with it. I’m sure that many people can see the same arbitrage opportunity and can capture it in a simple linear model. Because if it is a linear model, and you have the same arbitrage opportunity, practically everybody’s linear model is going to be very similar. There are not that many variations in the linear model in the field to capture a particular opportunity. And, therefore, because of this crowding, the power of a linear model can decay pretty rapidly. On the other hand, for a machine learning model, there are so many nuances and so many ways to construct it, and you can include so many different features, even though you know that, in this particular market, only certain features are relevant. But, there are many variations of particular features that you can add, and there are many ancillary features, perhaps from related markets, that you can add, which will make your model itself unique. I think that it is very unlikely that two persons’ machine learning models, even if they were to try to predict, let’s say, returns in the same market, will be similar—simply because of the large number of permutations of features that you can apply to your machine learning model. And, even with the same features, there are so many different machine learning algorithms, and so many ways to optimize the model or to select features that it is highly unlikely, even with the same set of features, that two researchers will produce the same model. So, I think that because of that, it is less likely to suffer over-crowding. But, on the flipside, that also increases the scope for over-fitting. But, with over-fitting, there are so many techniques nowadays to combat it, and we believe that those techniques are now finally able to bear fruit and eliminate the danger of over-fitting.
Steve Young: You touched on it very briefly: alpha, whether it is from an analysis or informational advantage, tends to decay over time as more people discover the source of alpha and crowd the trade. How do you stay ahead of the curve and continue generating alpha?
Dr. Ernest Chan: Well, by—talking machine learning. That’s the main tool that we try to combat it with because, like I said, no two persons can usually produce the same machine learning model. So, that’s one way. Another way is, following some of the research that Dr. Marcos Lopez de Prado has written, machine learning can be used for risk management as much as for generating predictions, for signal generation. So, we oftentimes have our own trading models that are linear, or very simple, or based on intuition, and it works okay over time. And, perhaps, lately you have suffered some market decay, so we would apply machine learning as a risk management layer to select trades that we think are particularly risky, and we would just avoid picking those trades. So, as a risk management layer, machine learning has the virtue that it does not create new trades, so it avoids some problems in over-fitting and, certainly, avoids some problems in alpha decay where it is basically used to select a subset of trades that our fundamental simple model is going to make. And that way, it has the benefits of both transparency and understandability, and, at the same time, of not giving up the power of the many other features that can be applied to the trading strategy. So, that’s the way that we try to avoid alpha decay—a combination of our simple model and a risk management layer, that can be very complicated and sophisticated, based on machine learning.
Steve Young: That’s a very interesting approach.
Do you think there is almost too much emphasis on the returns side of the equation and maybe not enough focus on risk management?
Dr. Ernest Chan: For a lot of smaller shops that seems to be the case. Certainly, for big funds, risk management have their own department. And, I don’t know how effective their risk management department is, but they certainly employ a lot of people and they certainly seem to be taking it seriously. But, for more smaller scale shops, I think perhaps risk management is more of an afterthought. And the reason I say that is that we can see that a lot of the smaller funds can blow up and just completely go out of business; some options-selling firms go out of business during 2018 and so forth. And, that is, to me, an indication that they did not really pay enough attention to risk management—perhaps, because there are not that many good tools for risk management available. A lot of the tools of risk management are unable to really go beyond a traditional value-at-risk kind of paradigm, and that’s really not sufficient for a lot of strategies. So, yes, I do believe risk management is a lower-hanging fruit. You know, with many strategies, if you can avoid a few big losses, and the strategy’s Sharpe ratio, general returns will be greatly improved. You do not need to find new strategies. You do not need to create new trades. Just the act of avoiding bad trades, could be really adding to the bottom line.
Steve Young: Yes! Very interesting! It’s a good perspective, and I totally agree with everything you said.
In a recent blog post of yours, you question whether news sentiment is still adding alpha. What are your thoughts on finding potential sources of information that might lead to alpha?
Dr. Ernest Chan: Well, news sentiment is a particular kind of alternative data that has been around for a long time, and there are multiple well-established firms offering such data, ranging from more traditional news providers, like Thompson Reuters and Bloomberg, to specialized boutique shops that process and sell news sentiment scores, and, therefore, it becomes in the view of a lot of researchers, and in our own experience, that they are adding less and less alpha. It is really difficult, but that is not to say that is true for all alternative data. There are new-fangled alternative data that are being created all the time and we constantly assess and partner with providers of such sources. Many times, they are not bound to add any alpha—that is true, but occasionally you find a diamond in the rough, and you hit the sweet spot of alternative data. The difficulty in this field is that unless you are a big shop, a multi-billion dollar fund, it is really hard to afford paying for all these alternative data because they are all very expensive. And, even if they are offered to you for free as a trial, you would have a hard time hiring enough people to analyze because alternative data takes a lot of effort to really put in the proper format and perform the due diligence on in order to make it suitable for machine learning. So, it’s not just the raw materials cost, but it is also the labour cost of turning this raw data into useable form. That is also very costly—in terms of labour. But, on the other hand, because it is so difficult that also is a barrier to entry. So, if you have some way of getting this data cheaply and finding the suitable researcher, the data scientist, to engineer this data into features then you would have a competitive advantage. So, anything difficult also has a flip side—, which also means that if you are good at it you have a competitive advantage. So that’s my view of alternative data: difficult, but, perhaps, if you can overcome it, it will lead to a unique source of alpha.
Steve Young: So, if alternative data is a potential source of alpha, does that necessarily mean that traditional—say, fundamental data, or macro-economic data, or more traditional financial data have, for the most part, been priced into asset prices? Or, do you think that with machine learning you can still use these more traditional sources of data and find, maybe, non-linear relationships or different combinations of data points that work well together? Do you think that is still possible or do you think really that it is all alternative data now?
Dr. Ernest Chan: I agree with you that actually one cannot say that the more traditional data, it’s been exhausted in terms of its ability to generate alpha. The idea is that, as you said, even for traditional data there are, perhaps, hundreds of possibilities. And, each data can be prophetic in many ways. You know, you can apply different technical indicators to them, you can apply different time-series techniques on them, and so forth. So, there are literally, perhaps, thousands of combinations that you can create from traditional data. And, from these thousands of, let’s say, time series of data, there are an astronomical number of combinations that you can create. So, machine learning is certainly useful even without alternative data because of that ability to explore a very large base of combinations of these data.
Steve Young: Alright!
Your books talk about a variety of assets classes from equities to FX and your firm systematically trades commodities and volatility. Your books also cover trading at various time horizons from monthly or weekly to high frequency. Where are you seeing potential sources of alpha in terms of trading time horizons, countries, or asset classes?
Dr. Ernest Chan: I think that for us the main alpha is to be found from holding a matter of hours to holding a matter of weeks. We are not a high frequency trading shop and we do not have the expertise and infrastructure, particularly, to really exploit millisecond frequency arbitrage opportunities. And, similarly, I think that we don’t have the tolerance for the possibly extended drawdowns that would necessarily accompany strategies that hold for months, hold positions for months, for example. So for us the sweet spot is intra-day trading strategy that holds from an hour to several hours to holding for, at most, a couple of weeks. Those are what we are comfortable with, but that is not to say that that is true for everybody. There are people who are able to exploit millisecond frequency opportunities. There are people who are comfortable with drawdowns. Like, if you are a pension fund, what is the big deal about a relatively severe drawdown? They are going to be here forever, so, maybe, they will see a value in investing in private equities, or real estate, or natural resources that they can hold for years and years and, maybe, suffer a couple of years when drawdown is not a big deal for them. And, ultimately, they may be able to extract a lot of return from that kind of long-term investment. And so, I would not say that my preference is necessarily universal—it’s just that for a small fund like ours that would be the sweet spot.
Steve Young: Do you think that, in terms of geographical location or asset classes, there are inefficiencies that can be taken advantage of?
Dr. Ernest Chan: Yes, I think that outside of the more liquid markets, like the US market, there may be more arbitrage opportunities, but, again, I think that one would need to have an expert in those markets to exploit that. It is not, oftentimes, as simple as just applying whatever works in the US markets and hope that it will work in another market. Because of the nuances in execution and regulations, oftentimes you do need to spend some time studying, gaining some expertise on that local market before the strategy can be translated there.
Steve Young: Do you think that equities are the hardest to find excess returns or do you think that other asset classes, like commodities or FX, are less efficient and it is easier to find excess returns there?
Dr. Ernest Chan: Well, we used to be equity traders—I used to be an equity trader, but, in the last, maybe, eight or nine years, I’ve gravitated towards forex and then, later on, futures and index instruments, and that is not because we didn’t pay attention to equities. We continued to do research in equities markets, but it seems to me that it is suddenly, especially for US equities, because of the problem of execution, because of the fragmentation of the trading venues in the US, it has become more and more difficult for us to generate short term alpha. Many people are aware of the high frequency trading activities that are going on in the US equity markets and they certainly impact the profitability of a lot of the intra-day equities strategies. And, so, we actually find that it has been indeed more difficult to find alpha in trading equities in our preferred time frame as opposed to trading futures and forex. So, that’s the reason we’ve been focussing on futures in the last, say, seven or eight years. It is simply because it seems that we are able to generate more of an alpha in those markets, which are less subject to the activities of high frequency trading.
Steve Young: Okay, that makes sense.
We started talking about machine learning already. Everyone is talking about machine learning these days. How do you see machine learning being applied in finance? And, what do you see as the pros and cons of machine learning over other quantitative techniques?
Dr. Ernest Chan: Well, as I said, I think that risk management is one big area that machine learning can really add a lot of value to finance. And, it has not been really put to good use in that particular area, if you look at published papers on machine learning most of them are about predicting returns on this or that, but not too many papers talk about risk management using machine learning, except for some rare, notable exceptions. So, that, I think, is its main direction. The problem of machine learning remains that the understandability is much lower, and transparency is low and, also, that it’s subject quite possibly to over-fitting, and it is very hard to detect. Any strategy could have a drawdown. If you have a machine learning strategy and it has a drawdown, nobody knows how to explain that drawdown and it becomes a very difficult management decision whether to continue it. So, it is the interaction of the machine learning and the human decision maker that is a little bit tricky. If the human decision makers are not able to understand machine learning well, that will create a problem. But, if you are using machine learning as risk management the worst you can do is to not make money, but it is unlikely that it would cause you to lose more money than you would have.
Steve Young: Okay. One of the big reasons, I would say, preventing the broader uptake of machine learning does seem to be that black box mentality: people are afraid of just trusting an algorithm without understanding the reasonings behind some of its conclusions. But, with modern interpretability techniques—the techniques out there that actually gain transparency into how the model uses the inputs to arrive at the outputs—do you think that as these become more mainstream and widely used that will help people accept machine learning models more?
Dr. Ernest Chan: Yes, I’m hoping that will help. We are in the process of experimenting with some of these and we will see if they really fulfill their promise. In academic papers, they look pretty good, but we will really field test them in the coming months to see if they really fulfill their promise in an actual commercial environment.
Steve Young: Okay. Sounds good!
That’s all the questions that I have. Are there any other comments that you would like to add?
Dr. Ernest Chan: No. I think those were very thought-provoking and excellent questions, and I’m glad you asked them.
Steve Young: Great! Thank you for that feedback.
We'd like to sincerely thank Dr. Chan for the thought provoking conversation.