Variable Metric Stochastic Approximation Theory
; Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, PMLR 5:560-566, 2009.
We provide a variable metric stochastic approximation theory. In doing so, we provide a convergence theory for a large class of online variable metric methods including the recently introduced online versions of the BFGS algorithm and its limited-memory LBFGS variant. We also discuss the implications of our results in the areas of elicitation of properties of distributions using prediction markets and in learning from expert advice.