Sparse Gaussian Conditional Random Fields: Algorithms, Theory, and Application to Energy Forecasting

Matt Wytock, Zico Kolter
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):1265-1273, 2013.

Abstract

This paper considers the sparse Gaussian conditional random field, a discriminative extension of sparse inverse covariance estimation, where we use convex methods to learn a high-dimensional conditional distribution of outputs given inputs. The model has been proposed by multiple researchers within the past year, yet previous papers have been substantially limited in their analysis of the method and in the ability to solve large-scale problems. In this paper, we make three contributions: 1) we develop a second-order active-set method which is several orders of magnitude faster that previously proposed optimization approaches for this problem 2) we analyze the model from a theoretical standpoint, improving upon past bounds with convergence rates that depend logarithmically on the data dimension, and 3) we apply the method to large-scale energy forecasting problems, demonstrating state-of-the-art performance on two real-world tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-wytock13, title = {Sparse Gaussian Conditional Random Fields: Algorithms, Theory, and Application to Energy Forecasting}, author = {Wytock, Matt and Kolter, Zico}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {1265--1273}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/wytock13.pdf}, url = {https://proceedings.mlr.press/v28/wytock13.html}, abstract = {This paper considers the sparse Gaussian conditional random field, a discriminative extension of sparse inverse covariance estimation, where we use convex methods to learn a high-dimensional conditional distribution of outputs given inputs. The model has been proposed by multiple researchers within the past year, yet previous papers have been substantially limited in their analysis of the method and in the ability to solve large-scale problems. In this paper, we make three contributions: 1) we develop a second-order active-set method which is several orders of magnitude faster that previously proposed optimization approaches for this problem 2) we analyze the model from a theoretical standpoint, improving upon past bounds with convergence rates that depend logarithmically on the data dimension, and 3) we apply the method to large-scale energy forecasting problems, demonstrating state-of-the-art performance on two real-world tasks.} }
Endnote
%0 Conference Paper %T Sparse Gaussian Conditional Random Fields: Algorithms, Theory, and Application to Energy Forecasting %A Matt Wytock %A Zico Kolter %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-wytock13 %I PMLR %P 1265--1273 %U https://proceedings.mlr.press/v28/wytock13.html %V 28 %N 3 %X This paper considers the sparse Gaussian conditional random field, a discriminative extension of sparse inverse covariance estimation, where we use convex methods to learn a high-dimensional conditional distribution of outputs given inputs. The model has been proposed by multiple researchers within the past year, yet previous papers have been substantially limited in their analysis of the method and in the ability to solve large-scale problems. In this paper, we make three contributions: 1) we develop a second-order active-set method which is several orders of magnitude faster that previously proposed optimization approaches for this problem 2) we analyze the model from a theoretical standpoint, improving upon past bounds with convergence rates that depend logarithmically on the data dimension, and 3) we apply the method to large-scale energy forecasting problems, demonstrating state-of-the-art performance on two real-world tasks.
RIS
TY - CPAPER TI - Sparse Gaussian Conditional Random Fields: Algorithms, Theory, and Application to Energy Forecasting AU - Matt Wytock AU - Zico Kolter BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-wytock13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 1265 EP - 1273 L1 - http://proceedings.mlr.press/v28/wytock13.pdf UR - https://proceedings.mlr.press/v28/wytock13.html AB - This paper considers the sparse Gaussian conditional random field, a discriminative extension of sparse inverse covariance estimation, where we use convex methods to learn a high-dimensional conditional distribution of outputs given inputs. The model has been proposed by multiple researchers within the past year, yet previous papers have been substantially limited in their analysis of the method and in the ability to solve large-scale problems. In this paper, we make three contributions: 1) we develop a second-order active-set method which is several orders of magnitude faster that previously proposed optimization approaches for this problem 2) we analyze the model from a theoretical standpoint, improving upon past bounds with convergence rates that depend logarithmically on the data dimension, and 3) we apply the method to large-scale energy forecasting problems, demonstrating state-of-the-art performance on two real-world tasks. ER -
APA
Wytock, M. & Kolter, Z.. (2013). Sparse Gaussian Conditional Random Fields: Algorithms, Theory, and Application to Energy Forecasting. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):1265-1273 Available from https://proceedings.mlr.press/v28/wytock13.html.

Related Material