Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition

Stephanie Milani, Anssi Kanervisto, Karolis Ramanauskas, Sander Schulhoff, Brandon Houghton, Sharada Mohanty, Byron Galbraith, Ke Chen, Yan Song, Tianze Zhou, Bingquan Yu, He Liu, Kai Guan, Yujing Hu, Tangjie Lv, Federico Malato, Florian Leopold, Amogh Raut, Ville Hautamäki, Andrew Melnik, Shu Ishida, João Henriques, Robert Klassert, Walter Laurito, Lucas Cazzonelli, Cedric Kulbach, Nicholas Popovic, Marvin Schweizer, Ellen Novoseller, Vinicius Goecks, Nicholas Waytowich, David Watkins, Josh Miller, Rohin Shah
Proceedings of the NeurIPS 2022 Competitions Track, PMLR 220:171-188, 2022.

Abstract

To facilitate research in the direction of fine-tuning foundation models from human feedback, we held the MineRL BASALT Competition on Fine-Tuning from Human Feedback at NeurIPS 2022. The BASALT challenge asks teams to compete to develop algorithms to solve tasks with hard-to-specify reward functions in Minecraft. Through this competition, we aimed to promote the development of algorithms that use human feedback as channels to learn the desired behavior. We describe the competition and provide an overview of the top solutions. We conclude by discussing the impact of the competition and future directions for improvement.

Cite this Paper

BibTeX
@InProceedings{pmlr-v220-milani23a, title = {Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition}, author = {Milani, Stephanie and Kanervisto, Anssi and Ramanauskas, Karolis and Schulhoff, Sander and Houghton, Brandon and Mohanty, Sharada and Galbraith, Byron and Chen, Ke and Song, Yan and Zhou, Tianze and Yu, Bingquan and Liu, He and Guan, Kai and Hu, Yujing and Lv, Tangjie and Malato, Federico and Leopold, Florian and Raut, Amogh and Hautam\"aki, Ville and Melnik, Andrew and Ishida, Shu and Henriques, Jo\~ao and Klassert, Robert and Laurito, Walter and Cazzonelli, Lucas and Kulbach, Cedric and Popovic, Nicholas and Schweizer, Marvin and Novoseller, Ellen and Goecks, Vinicius and Waytowich, Nicholas and Watkins, David and Miller, Josh and Shah, Rohin}, booktitle = {Proceedings of the NeurIPS 2022 Competitions Track}, pages = {171--188}, year = {2022}, editor = {Ciccone, Marco and Stolovitzky, Gustavo and Albrecht, Jacob}, volume = {220}, series = {Proceedings of Machine Learning Research}, month = {28 Nov--09 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v220/milani23a/milani23a.pdf}, url = {https://proceedings.mlr.press/v220/milani23a.html}, abstract = {To facilitate research in the direction of fine-tuning foundation models from human feedback, we held the MineRL BASALT Competition on Fine-Tuning from Human Feedback at NeurIPS 2022. The BASALT challenge asks teams to compete to develop algorithms to solve tasks with hard-to-specify reward functions in Minecraft. Through this competition, we aimed to promote the development of algorithms that use human feedback as channels to learn the desired behavior. We describe the competition and provide an overview of the top solutions. We conclude by discussing the impact of the competition and future directions for improvement.} }
Endnote
%0 Conference Paper %T Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition %A Stephanie Milani %A Anssi Kanervisto %A Karolis Ramanauskas %A Sander Schulhoff %A Brandon Houghton %A Sharada Mohanty %A Byron Galbraith %A Ke Chen %A Yan Song %A Tianze Zhou %A Bingquan Yu %A He Liu %A Kai Guan %A Yujing Hu %A Tangjie Lv %A Federico Malato %A Florian Leopold %A Amogh Raut %A Ville Hautamäki %A Andrew Melnik %A Shu Ishida %A João Henriques %A Robert Klassert %A Walter Laurito %A Lucas Cazzonelli %A Cedric Kulbach %A Nicholas Popovic %A Marvin Schweizer %A Ellen Novoseller %A Vinicius Goecks %A Nicholas Waytowich %A David Watkins %A Josh Miller %A Rohin Shah %B Proceedings of the NeurIPS 2022 Competitions Track %C Proceedings of Machine Learning Research %D 2022 %E Marco Ciccone %E Gustavo Stolovitzky %E Jacob Albrecht %F pmlr-v220-milani23a %I PMLR %P 171--188 %U https://proceedings.mlr.press/v220/milani23a.html %V 220 %X To facilitate research in the direction of fine-tuning foundation models from human feedback, we held the MineRL BASALT Competition on Fine-Tuning from Human Feedback at NeurIPS 2022. The BASALT challenge asks teams to compete to develop algorithms to solve tasks with hard-to-specify reward functions in Minecraft. Through this competition, we aimed to promote the development of algorithms that use human feedback as channels to learn the desired behavior. We describe the competition and provide an overview of the top solutions. We conclude by discussing the impact of the competition and future directions for improvement.
APA
Milani, S., Kanervisto, A., Ramanauskas, K., Schulhoff, S., Houghton, B., Mohanty, S., Galbraith, B., Chen, K., Song, Y., Zhou, T., Yu, B., Liu, H., Guan, K., Hu, Y., Lv, T., Malato, F., Leopold, F., Raut, A., Hautamäki, V., Melnik, A., Ishida, S., Henriques, J., Klassert, R., Laurito, W., Cazzonelli, L., Kulbach, C., Popovic, N., Schweizer, M., Novoseller, E., Goecks, V., Waytowich, N., Watkins, D., Miller, J. & Shah, R.. (2022). Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition. Proceedings of the NeurIPS 2022 Competitions Track, in Proceedings of Machine Learning Research 220:171-188 Available from https://proceedings.mlr.press/v220/milani23a.html.

Related Material