A Stochastic Differential Equation Framework for Guiding Online User Activities in Closed Loop

Yichen Wang, Evangelos Theodorou, Apurv Verma, Le Song
Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, PMLR 84:1077-1086, 2018.

Abstract

Recently, there is a surge of interest in using point processes to model continuous-time user activities. This framework has resulted in novel models and improved performance in diverse applications. However, most previous works focus on the ”open loop” setting where learned models are used for predictive tasks. Typically, we are interested in the ”closed loop” setting where a policy needs to be learned to incorporate user feedbacks and guide user activities to desirable states. Although point processes have good predictive performance, it is not clear how to use them for the challenging closed loop activity guiding task. In this paper, we propose a framework to reformulate point processes into stochastic differential equations, which allows us to extend methods from stochastic optimal control to address the activity guiding problem. We also design an efficient algorithm, and show that our method guides user activities to desired states more effectively than state-of-arts.

Cite this Paper

BibTeX
@InProceedings{pmlr-v84-wang18d, title = {A Stochastic Differential Equation Framework for Guiding Online User Activities in Closed Loop}, author = {Wang, Yichen and Theodorou, Evangelos and Verma, Apurv and Song, Le}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {1077--1086}, year = {2018}, editor = {Storkey, Amos and Perez-Cruz, Fernando}, volume = {84}, series = {Proceedings of Machine Learning Research}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/wang18d/wang18d.pdf}, url = {https://proceedings.mlr.press/v84/wang18d.html}, abstract = {Recently, there is a surge of interest in using point processes to model continuous-time user activities. This framework has resulted in novel models and improved performance in diverse applications. However, most previous works focus on the ”open loop” setting where learned models are used for predictive tasks. Typically, we are interested in the ”closed loop” setting where a policy needs to be learned to incorporate user feedbacks and guide user activities to desirable states. Although point processes have good predictive performance, it is not clear how to use them for the challenging closed loop activity guiding task. In this paper, we propose a framework to reformulate point processes into stochastic differential equations, which allows us to extend methods from stochastic optimal control to address the activity guiding problem. We also design an efficient algorithm, and show that our method guides user activities to desired states more effectively than state-of-arts.} }
Endnote
%0 Conference Paper %T A Stochastic Differential Equation Framework for Guiding Online User Activities in Closed Loop %A Yichen Wang %A Evangelos Theodorou %A Apurv Verma %A Le Song %B Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2018 %E Amos Storkey %E Fernando Perez-Cruz %F pmlr-v84-wang18d %I PMLR %P 1077--1086 %U https://proceedings.mlr.press/v84/wang18d.html %V 84 %X Recently, there is a surge of interest in using point processes to model continuous-time user activities. This framework has resulted in novel models and improved performance in diverse applications. However, most previous works focus on the ”open loop” setting where learned models are used for predictive tasks. Typically, we are interested in the ”closed loop” setting where a policy needs to be learned to incorporate user feedbacks and guide user activities to desirable states. Although point processes have good predictive performance, it is not clear how to use them for the challenging closed loop activity guiding task. In this paper, we propose a framework to reformulate point processes into stochastic differential equations, which allows us to extend methods from stochastic optimal control to address the activity guiding problem. We also design an efficient algorithm, and show that our method guides user activities to desired states more effectively than state-of-arts.
APA
Wang, Y., Theodorou, E., Verma, A. & Song, L.. (2018). A Stochastic Differential Equation Framework for Guiding Online User Activities in Closed Loop. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 84:1077-1086 Available from https://proceedings.mlr.press/v84/wang18d.html.

Related Material