Descent Methods for Tuning Parameter Refinement

Alexander Lorbert, Peter Ramadge
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:469-476, 2010.

Abstract

This paper addresses multidimensional tuning parameter selection in the context of “train-validate-test” and $K$-fold cross validation. A coarse grid search over tuning parameter space is used to initialize a descent method which then jointly optimizes over variables and tuning parameters. We study four regularized regression methods and develop the update equations for the corresponding descent algorithms. Experiments on both simulated and real-world datasets show that the method results in significant tuning parameter refinement.

Cite this Paper

BibTeX
@InProceedings{pmlr-v9-lorbert10a, title = {Descent Methods for Tuning Parameter Refinement}, author = {Lorbert, Alexander and Ramadge, Peter}, booktitle = {Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics}, pages = {469--476}, year = {2010}, editor = {Teh, Yee Whye and Titterington, Mike}, volume = {9}, series = {Proceedings of Machine Learning Research}, address = {Chia Laguna Resort, Sardinia, Italy}, month = {13--15 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v9/lorbert10a/lorbert10a.pdf}, url = {https://proceedings.mlr.press/v9/lorbert10a.html}, abstract = {This paper addresses multidimensional tuning parameter selection in the context of “train-validate-test” and $K$-fold cross validation. A coarse grid search over tuning parameter space is used to initialize a descent method which then jointly optimizes over variables and tuning parameters. We study four regularized regression methods and develop the update equations for the corresponding descent algorithms. Experiments on both simulated and real-world datasets show that the method results in significant tuning parameter refinement.} }
Endnote
%0 Conference Paper %T Descent Methods for Tuning Parameter Refinement %A Alexander Lorbert %A Peter Ramadge %B Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2010 %E Yee Whye Teh %E Mike Titterington %F pmlr-v9-lorbert10a %I PMLR %P 469--476 %U https://proceedings.mlr.press/v9/lorbert10a.html %V 9 %X This paper addresses multidimensional tuning parameter selection in the context of “train-validate-test” and $K$-fold cross validation. A coarse grid search over tuning parameter space is used to initialize a descent method which then jointly optimizes over variables and tuning parameters. We study four regularized regression methods and develop the update equations for the corresponding descent algorithms. Experiments on both simulated and real-world datasets show that the method results in significant tuning parameter refinement.
RIS
TY - CPAPER TI - Descent Methods for Tuning Parameter Refinement AU - Alexander Lorbert AU - Peter Ramadge BT - Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics DA - 2010/03/31 ED - Yee Whye Teh ED - Mike Titterington ID - pmlr-v9-lorbert10a PB - PMLR DP - Proceedings of Machine Learning Research VL - 9 SP - 469 EP - 476 L1 - http://proceedings.mlr.press/v9/lorbert10a/lorbert10a.pdf UR - https://proceedings.mlr.press/v9/lorbert10a.html AB - This paper addresses multidimensional tuning parameter selection in the context of “train-validate-test” and $K$-fold cross validation. A coarse grid search over tuning parameter space is used to initialize a descent method which then jointly optimizes over variables and tuning parameters. We study four regularized regression methods and develop the update equations for the corresponding descent algorithms. Experiments on both simulated and real-world datasets show that the method results in significant tuning parameter refinement. ER -
APA
Lorbert, A. & Ramadge, P.. (2010). Descent Methods for Tuning Parameter Refinement. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 9:469-476 Available from https://proceedings.mlr.press/v9/lorbert10a.html.

Related Material