Machine Learning And The Future Of Financial Services - An Interview With Prof. John Hull

The following transcript is from the conversation between Steve Young of ENJINE and Prof. John Hull.

Among quants, Prof. Hull needs no introduction. He is Maple Financial Professor of Derivatives and Risk Management and Academic Director, Rotman Financial Innovation Hub at Rotman School of Management. He is best known for his books 'Risk Management and Financial Institutions', 'Options, Futures, and Other Derivatives', and 'Fundamentals of Futures and Options Markets'. His books are widely used among traders and students alike.

Steve Young: Professor Hull, you are very well-known for your books on derivatives valuation and risk management, the Hull-White interest rate model, what led you to writing your latest book Machine Learning in Business?

Professor Hull: I’ve enjoyed my research in derivatives a lot. Moving from derivatives to ML is not quite as big a leap as you might think. We use algorithms in derivatives and I’ve had to develop expertise in that area over time. Some of that expertise carries over to ML. And, of course, there’s a lot of applications of ML in derivatives and risk management and those are what I’ve concentrated on as far as my research is concerned. But, of course, my teaching is concerned with a much broader range of applications.

Steve Young: I’ve read your latest book and I think it’s a really good introduction to machine learning just for anyone in general. Who’s the target audience for this book and what will they take away from reading it?

Professor Hull: Well, at Rotman, we now teach ML on all our programs. We of course teach ML on our Master of Management Analytics (MMA) program. The Master of Finance (MFin) and Master of Financial Risk Management (MFRM) programs now have compulsory courses on ML. The MBA and the undergraduate BCom students have an ML elective.

I needed a textbook for the ML courses I teach and looked at a number of alternatives. I found that there are two categories of books out there. First, there are books that are written by computer scientists for computer scientists. I have learned a lot from those books, but they’re not really suitable for business students. The sort of nitty-gritty detail that computer scientists are interested in is not always what business students need. The second category of books is at the other extreme. They never talk about any of the algorithms at all. They explain that ML is going to change the world, but give hardly any details about how it’s going to do that. So I thought that there was a need for something in the middle for our students. They need to understand the most popular algorithms and the successful business applications that are out there. They do not need to worry about convergence issues, how run time increases with the number of observations, and so on. Also, I find that it is best to avoid matrix and vector algebra when explaining ideas to business students.

The objective of my book is not to convert the reader into a data scientist. Some of my students do get so interested in this area that they actually want to become data scientists, but that is the minority. The objective of the book is really to put them in a position where they know enough about what data scientists do that they can work productively with them.

So I wanted a book that was somewhere between the two extremes that I mentioned. The students I teach have got 30 to 40 years ahead of them in the financial sector. They will not survive unless they can work productively with data scientists. Like my other books in derivatives, I would like to think that book is useful to practitioners as well as students. In fact, it’s been selling well with practitioners. Typically practitioner readers of my book recognize the way the world’s moving and want to know a little bit more about what data science is all about.

People who can act as intermediaries between the data scientists and the rest of an organization are key employees and hopefully the readers of my book can become such intermediaries.

Steve Young: I think it is really good training for a business-type person to get a really good exposure to, and a pretty solid foundation, actually, in ML. It gives enough detail whereby you can communicate pretty intelligently with a data scientist.

Professor Hull: Yes. I’d like to think that is true. Of course ML is a fast changing field and so I will have to work hard to keep the book up to date. Natural language processing is an area where a lot of progress is being made. Another important development within ML is the quest to move away from black box algorithms toward output that is more transparent and interpretable.

Steve Young: Yes! And that’s going to lead us to some more questions we’ll get to later on. As you’ve kind of eluded to, business and finance can be slow to adopt new technologies and methodologies.

What do you think is behind some of the hesitation around the uptake of ML? Are they technical issues or are they some of the societal issues that were in your book? And how do we address them?

Professor Hull: I think if you go back five or ten years, there was quite a lot of resistance to ML. I don’t see that now in the financial sector. Some financial institutions are playing catch up, investing huge resources in ML. Sometimes ML projects are not evaluated in the same way as other projects. They are simply regarded as something that has got to be done. So I don’t know that I agree with you that there is a lot of hesitation. I think that there used to be, but now most of the financial institutions that I talk to have overcome that hesitation. Some of them have set up separate units away from the main organization to pursue ML implementations; others are working closely with fintech start ups. Of course many financial institutions are huge with hundreds of thousands of employees. Changing the culture is like turning round the Titanic. So there may be resistance from employees but I think there is a clear realization at the top that ML applications are important.

Steve Young: What do you think made that shift five to ten years ago? Was it a fear of missing out or was there more evidence that ML capabilities were superior? Or, were there new techniques that overcame fears—like the transparency and interpretability techniques that you mentioned?

Professor Hull: Well, I think probably all of those. I think the fear of being left behind is always a worry. Different financial institutions that I talk to have tried different approaches as I mentioned earlier. Some have set up separate autonomous units away from the main operations. Others have bought start ups in order to acquire the necessary expertise. I think there are now enough success stories for ML that everyone realizes that it has to be taken seriously.

Steve Young: Amongst the big financial institutions that are adopting ML, are they applying it more for customer analytics or marketing, or are they applying it more to the capital markets, or risk management, or on the trading floor for trading strategies? Where do you see the applications?

Professor Hull: All of those really. There are important applications in marketing. Unsupervised learning can enable you to understand your customers better and communicate with them better. When you have data on hundreds of thousands or millions of customers, a machine can cluster customers into meaningful groups better than a human being. This is important. For example, one cluster of a bank’s customers might be candidates for a mortgage, another for wealth management, and so on.

Credit decisions are where ML has had a huge impact. A machine can decide whether a loan should be made as well or better than a human being. Fraud detection is increasingly being done by ML algorithms. Money laundering is being detected by ML algorithms. Dealing with all the different regulatory environments throughout the world is often handled by ML algorithms. Even personnel decisions have been impacted. If you send your CV in to a large financial institution, the chances are it will at least be filtered by an ML algorithm. The valuation and risk management of derivatives has been impacted by ML. There’s quite a number of groups throughout the world—including a team that I’m working with—that are trying to apply reinforcement learning to hedging decisions. Neural networks can be used to provide fast pricing of exotic options for a range of different models. So I wouldn’t want to focus on one particular area of application. I think that pretty much every part of a financial institution is going to be impacted by ML in the next few years.

Steve Young: I remember the example you gave in your book about using ML for derivatives valuation essentially in replacement of a Monte Carlo. I thought that was a very interesting application.

Professor Hull: Yes. It is. My research team at Rotman is working with a start up, Riskfuel, doing exactly this. Some methods for valuing derivatives such as Monte Carlo simulation are quite slow. But they can be replicated with a neural network and, once this has been done, the valuation is really fast. This can be important for scenario analyses and other activities that are carried out in connection with a derivative dealer’s portfolio.

Steve Young: Cool! So for those in quantitative finance who are accustomed to more traditional statistical or econometric techniques—for example, for trading or portfolio management—what are some ML tools and techniques best suited to those tasks and what are the advantages and disadvantages of ML relative to more traditional techniques?

Professor Hull: Well, one of the things about ML, of course, is that it relies on data. So, whatever you’re talking about, you’ve got to have a relevant data set. If the world’s changing so that past data is no longer applicable to whatever decision you’re making, then ML is not going to be helpful. That’s why ML has been very successful with credit decisions—because assessing whether somebody will default on a loan today is much the same as assessing whether someone would default on a loan ten years ago. The criteria used are very similar.

As far as investment decisions are concerned, you may come up with a strategy that’s worked well in the past, but perhaps the dynamics of markets have changed. Perhaps a trading strategy that worked well in the past is no longer profitable because it has become well known. But, having said that, companies like Renaissance Technologies have demonstrated the power of ML to find patterns in historical data that are likely to be repeated. The repetition does not have to be a sure thing. All we need is that a particular trading strategy has a more than 50% chance of being successful. Renaissance Technologies were 20 years ahead of their time in their use of ML.

The fact that ML relies on data is one of the reasons why its importance will continue to increase. I’m sure you’ve heard statistics on the way in which the data existing in the world is increasing exponentially. For example, 90% of available data was created in the last two years. If the data available in the world is increasing exponentially, it is reasonable to suppose that ML applications will increase exponentially.

Steve Young: A comment that I’ve heard before is that people are happy with their linear model, their linear regressions, and, as you mentioned, there is an explosion of data. Given the same data, would an ML model be able to extract additional patterns or pick up on extra signals that a linear model wouldn’t be able to, or would we be able to incorporate some sort of, maybe, a macro indicator to identify some sort of regime shift, as you mentioned? Are ML models more capable of detecting stuff like that versus a linear model?

Professor Hull: Linear models have the advantage that they require very little data. Once you have decided that a certain relationship is linear a few hundred data items might be enough to determine the relationship. But the world’s not totally linear. One of the big advantages of ML models is that they can be used to determine a non-linear relationship. Decision trees and neural networks are good at doing this. But to determine a non-linear relationship you need lots of data. We can now do things that were not possible in the past. Computers are faster and as mentioned we have more data than ever before. So linear models will continue to be used, but ML gives rise to a richer set of possibilities.

Steve Young: Does that go back to your point about five to ten years ago there seems to be an acceptance of ML? Was there a kind of a tipping point in terms of data as well?

Professor Hull: Well, I don’t know exactly when the tipping point was. To be honest with you, I knew virtually nothing about ML five to ten years ago—like most people. Then in the last three to four years it seems as though, whenever you open a newspaper or a magazine, there is an article about some aspect of AI or ML. So the world seems to have changed, and, as I mentioned earlier, we at Rotman have tried to change with it.

If someone had suggested, maybe even only five years ago, that we really ought to run an elective in ML for our students, I’d think, there would not have been much support. Now, everyone is talking about ML and we cannot ignore it. Finance, marketing, and accounting researchers all recognize that ML is the way of the future. Indeed one of my accounting colleagues recently carried out an ML analysis of the ranking of business schools which I thought was very interesting. So Rotman is actually applying ML to its own business!

Steve Young: That’s interesting. So Rotman is one of the best business schools in the world and the University of Toronto just happens to be one of the major powerhouses in terms of AI research, especially with Geoff Hinton. Is there any synergy there? Do you guys communicate at all?

Professor Hull: Yes, we do. Geoff Hinton has been a great ML pioneer. Also, we have a particularly close relationship with the engineering faculty. I’m Academic Director of FinHub and one of the things it does is arrange summer internships for students.

What we’ve been trying to do, and it’s been quite successful, is pair a Rotman student with an engineering student to do a summer internship at a financial institution. The Rotman student has more business skills and the engineering student has more technical skills. On that note, one thing we have found is that students on our two-year MBA program need a ML elective in the first year, not the second year, so that they can apply for a wider range of internship opportunities between the two years. We are addressing that by offering my elective in the first year of the MBA program.

Steve Young: Are you seeing a lot of demand for, say, a finance practitioner with ML background, and is there a supply imbalance there?

Professor Hull: Absolutely. If you go back five or ten years, what did a student need to have on their resume to be taken seriously in a job interview? Excel might be enough. Today, you’ve got to have coding, ideally Python, on your resume to get a job in finance. If you can indicate that you’ve done an ML project or taken an ML elective that helps.

When I talk to the human resources people at financial institutions they tell me that coding is a must. That didn’t used to be the case. One of the things that students who take my courses have got to do is learn Python. Some already use Python and so they have an advantage, but at the other extreme are students who have done no coding at all. Learning Python is not a hard sell to students today. It is well documented that large financial institutions in the US—like JP Morgan and Goldman Sachs—insist that their new hires learn Python. Luckily, many students pick up coding skills in high school these days and so we can expect the coding skills of Rotman students to improve over time.

Steve Young: Do you see ML as a complement or a substitute for traditional statistical techniques? For example, do we still need to worry about having independent and identically distributed observations or serial correlation or statistical significance?

Professor Hull: Of course it is always nice to have independent and identically distributed observations whatever you’re doing. But ML does not rely on this. Many people have taken a statistics course at some time in their studies. I think statistics courses are changing. The traditional material on probability distributions, significance tests, confidence intervals and linear regression is still important. But I would say that going forward at least half of any statistics course should be concerned with machine learning algorithms. Of course, not everyone would agree with me and that means that in the university environment we will probably change slowly.

There is a difference between the traditional way of thinking and the ML way of thinking about the world. Traditionally, if you are an economist or a statistician you come up with a hypothesis, and then you take the hypothesis to the data and you test it. ML is different because everything just comes from the data. There’s no hypothesis in ML. People who have been brought up with the traditional way of doing research may find it difficult to accept research that is totally data driven without any hypothesis.

Steve Young: Great! So is that a bit of a paradigm shift—where we’re relying on the algorithm in essence to detect the patterns instead of us coming up with a hypothesis?

Professor Hull: I think “paradigm shift” describes it well. There is a paradigm shift going from a hypothesis-driven world to a purely data-driven world. And, you can imagine, if your whole career has been invested in traditional types of research and teaching, it might be difficult to make the transition.

Steve Young: You mentioned this earlier, given a lot of excitement around ML, it’s capabilities to incorporate far more data, ability to capture non-linear relationships, relationships between features, and there are numerous cases, many high profile, where ML performed spectacularly well, there have been some applications of ML that haven’t quite lived up to expectation—such as AI powered ETFs, or hedge funds that are underperforming, or highly-refined traditional models that are still winning the M Competitions for forecasting. Why do you think ML has not performed as well in those applications?

Professor Hull: Well, I think we touched on some of this earlier. If the data you are using is not relevant to the decision being made, ML will fail. You can have lots and lots of data, but, if the world’s changing and the data’s not relevant to the decision being made, then ML won’t work. Also, data cleaning has to be handled well. It is estimated that data cleaning, which includes making sure that only relevant data is retained, is 80% of a data scientist’s work. It is not the most exciting part of what a data scientist does, but if it is not done well the results from an ML project will suffer.

Steve Young: Is there a danger of people blindly applying an ML algorithm to a set of data without the necessary background or expertise, and maybe detecting a false pattern or a false relationship?

Professor Hull: I think there is. It is tempting to bring a data set to an ML package without any real knowledge of how the package works and how the parameters of the package should be set. If a user does not like the results from one package it is almost too easy just to try another one without thinking too much about what is being done. That’s why I think it’s important that people have some understanding of algorithms together with their strengths and weaknesses. That is what I try to cover in my book.

Steve Young: Is there perhaps—do we need more of a bridge between the traditional statisticians and data scientists, and I‘ll give you a bit of an example. ML experts are big fans of cross validation, for example, and they think that ten folds of cross validation is enough folds to have that confidence that the model will perform sufficiently well out of sample. But a traditional statistician would probably say ten observations is not statistically significant. Do we need to bridge that gap and get the data scientists to recognize that there are statistical rigours?

Professor Hull: One of the things that I stress to students is the importance of dividing data into a training set, validation set, and test set. Students do three major projects and they will not get good grades if they fail to do this. The importance of testing models out of sample is something statisticians have realized for a long time. In ML we are often trying out many different models. Hence the need for two out-of-sample data sets. I have both baby examples and bigger examples in my book to illustrate this.

Steve Young: It does seem like there’s a movement more towards just out of sample performance of the model, being able to make accurate predictions, instead of, for example, with linear regressions, it’s all about being able to explain the variance of the data.

Professor Hull: That is right. Prediction is not a big part of traditional statistics, which is more concerned with significance tests and confidence intervals, and so on. But when you are predicting something, you obviously want to make sure that whatever rule you have come up with actually generalizes well to the validation set. And if you are looking at lots of different rules, then you’ve got to use a test set to make sure you’ve not cheated and, say, looked at a hundred different rules to find one that works really well.

Steve Young: Exactly, yes.

Professor Hull: You’ve got to be very careful about this, because some of the models that we use in ML, have got literally tens of thousands of parameters, and with that number of parameters you have to be really careful to avoid overfitting.

Steve Young: Yes. Over fitting is the big danger with these models.

Professor Hull: Absolutely.

Steve Young: So what do you see on the horizon for ML in business? Where and how will ML be used? How will ML evolve to better suit their needs?

Professor Hull: Well, I see more and more applications for ML in business. What I tell my students is all areas within financial services are likely to be heavily impacted by ML over the next 10 years—and that this certainly going to be the case during the lifetime of a student’s career. So I wouldn’t necessarily want to pinpoint one particular area, but I think it’s important to develop skills in managing large data sets and updating large data sets because large data sets are going to drive so many different innovations going forward.

I think, if I were starting out again, I would want to be a data scientist, because, looking ahead, data science is just going to be so important, not just for financial institutions, but for organizations of all types. Remember the 90% rule. 90% of the available data has been created in the last two years. That is likely to continue to be true. Also, computers are going to continue to get faster. Just think what effect quantum computing is going to have on ML algorithms—in terms of their ability to handle large volumes of data effectively.

As mentioned earlier, I think lots of progress will continue to be made in natural language processing. Transparency and interpretability will continue to receive attention. Let’s take a simple example. If you go to a financial institution for a loan, and it turns you down, you are likely to ask a representative why this has happened. The answer “It was this algorithm” is not likely to impress you. The European Union rules on data include a “right to explanation” clause, but this is common sense requirement as well

Artificial intelligence is the fourth industrial revolution. A lot of people talk about jobs being lost, and, certainly, in some areas jobs will change as they do in all industrial revolutions. But, as I say in my book, job losses have been forecast for every previous industrial revolution and it has not happened. Of course previous industrial revolutions have led to some types of jobs disappearing. But as a human race we have shown a great capacity to create more jobs so that we achieve almost full employment.

Steve Young: Great! That’s all the questions that I have. Is there anything that you’d like to add?

Professor Hull: Not really. I think your questions were the important ones. ML is an exciting area, and I would advise everybody who’s in their 20s or 30s to learn as much about this area as possible because there is no question that it is going to become increasingly important. Learn coding. Get involved in an ML project if you get the chance. Do not forget the importance of continually updating your skills in the fast changing world we live in.

We'd like to sincerely thank Prof. Hull for his time. For more information on his new book "Machine Learning in Business", please follow this link.