A Reinforcement Learning Based Approach to Pricing and Hedging Financial Derivatives

Abstract

Options pricing is a notoriously tricky subject to maneuver, both mathematically and empirically. How- ever, the rise of reinforcement learning in recent times has opened up a new avenue to pricing options through machine learning. As a result, this paper explores and extends the discrete-time option pricing model by Halperin [11]. More specifically, we focus on the off-policy algorithm well known as the Q-Learning algorithm. In order to transform the original problem of pricing a Black Scholes based derivative, we stray from the academic continuous-time limit and use a discretized version of the classic Black Scholes model in order to construct a risk-neutral Markov Decision Process. Here, the option price of the MDP is an optimal Q-function. Pricing and hedging an option when the stock dynamics - such as the volatility - are unknown is now possible through the use of the Q-Learning algorithm. The model itself does not make any guesses or assumptions about the structure of the data it is given, and is guaranteed to asymptotically converge to the optimal solution given enough data and time. Empirical results show that the pricing of European options is very close to the classical Black-Scholes results, although there are some notable differences and error deviations when controlling for different factors. This model can also be easily extended to practically any type of financial option, given an easily tractable or estimated payoff function. Extensive numerical testing is done, and specific recommendations of further research are given.