Deep Reinforcement Learning for High-Frequency Market Making

Pankaj Kumar
Proceedings of The 14th Asian Conference on Machine Learning, PMLR 189:531-546, 2023.

Abstract

High-frequency market making is a algorithmic trading strategy in which an agent provides liquidity at the same time as quoting a bid price and an ask price on a security. The strategy reap profits in the form of the spread between the quoted price placed on the buy and sell prices. Due to complexity in inventory risk, counterparties to trades and information asymmetry, the understanding of high-frequency market making algorithms is relatively unexplored by academics across disciplines. In this paper, we develop realistic simulations of limit order markets and use them to design a high-frequency market making agent using Deep Recurrent Q-Networks. Our approach outperforms a prominent benchmark strategy from literature, which uses temporal-difference reinforcement learning to design market making agents. Using the simulation framework, we analyse how the maker-take fee, a feature of market design, affects market quality and the agent’s profitability. The agents successfully reproduce stylised facts in historical trade data from each simulation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v189-kumar23a, title = {Deep Reinforcement Learning for High-Frequency Market Making}, author = {Kumar, Pankaj}, booktitle = {Proceedings of The 14th Asian Conference on Machine Learning}, pages = {531--546}, year = {2023}, editor = {Khan, Emtiyaz and Gonen, Mehmet}, volume = {189}, series = {Proceedings of Machine Learning Research}, month = {12--14 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v189/kumar23a/kumar23a.pdf}, url = {https://proceedings.mlr.press/v189/kumar23a.html}, abstract = {High-frequency market making is a algorithmic trading strategy in which an agent provides liquidity at the same time as quoting a bid price and an ask price on a security. The strategy reap profits in the form of the spread between the quoted price placed on the buy and sell prices. Due to complexity in inventory risk, counterparties to trades and information asymmetry, the understanding of high-frequency market making algorithms is relatively unexplored by academics across disciplines. In this paper, we develop realistic simulations of limit order markets and use them to design a high-frequency market making agent using Deep Recurrent Q-Networks. Our approach outperforms a prominent benchmark strategy from literature, which uses temporal-difference reinforcement learning to design market making agents. Using the simulation framework, we analyse how the maker-take fee, a feature of market design, affects market quality and the agent’s profitability. The agents successfully reproduce stylised facts in historical trade data from each simulation.} }
Endnote
%0 Conference Paper %T Deep Reinforcement Learning for High-Frequency Market Making %A Pankaj Kumar %B Proceedings of The 14th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Emtiyaz Khan %E Mehmet Gonen %F pmlr-v189-kumar23a %I PMLR %P 531--546 %U https://proceedings.mlr.press/v189/kumar23a.html %V 189 %X High-frequency market making is a algorithmic trading strategy in which an agent provides liquidity at the same time as quoting a bid price and an ask price on a security. The strategy reap profits in the form of the spread between the quoted price placed on the buy and sell prices. Due to complexity in inventory risk, counterparties to trades and information asymmetry, the understanding of high-frequency market making algorithms is relatively unexplored by academics across disciplines. In this paper, we develop realistic simulations of limit order markets and use them to design a high-frequency market making agent using Deep Recurrent Q-Networks. Our approach outperforms a prominent benchmark strategy from literature, which uses temporal-difference reinforcement learning to design market making agents. Using the simulation framework, we analyse how the maker-take fee, a feature of market design, affects market quality and the agent’s profitability. The agents successfully reproduce stylised facts in historical trade data from each simulation.
APA
Kumar, P.. (2023). Deep Reinforcement Learning for High-Frequency Market Making. Proceedings of The 14th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 189:531-546 Available from https://proceedings.mlr.press/v189/kumar23a.html.

Related Material