- title: 'NeurIPS 2022 Competition Track Revised Selected Papers'
abstract: 'Introduction to this volume.'
volume: 220
URL: https://proceedings.mlr.press/v220/ciccone23a.html
PDF: https://proceedings.mlr.press/v220/ciccone23a/ciccone23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-ciccone23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: i-i
id: ciccone23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: i
lastpage: i
published: 2023-08-31 00:00:00 +0000
- title: 'Lessons Learned from Ariel Data Challenge 2022 - Inferring Physical Properties of Exoplanets From Next-Generation Telescopes'
abstract: 'Exo-atmospheric studies, i.e. the study of exoplanetary atmospheres, is an emerging frontier in Planetary Science. To understand the physical properties of hundreds of exoplanets, astronomers have traditionally relied on sampling-based methods. However, with the growing number of exoplanet detections (i.e. increased data quantity) and advancements in technology from telescopes such as JWST and Ariel (i.e. improved data quality), there is a need for more scalable data analysis techniques. The Ariel Data Challenge 2022 aims to find interdisciplinary solutions from the NeurIPS community. Results from the challenge indicate that machine learning (ML) models have the potential to provide quick insights for thousands of planets and millions of atmospheric models. However, the machine learning models are not immune to data drifts, and future research should investigate ways to quantify and mitigate their negative impact.'
volume: 220
URL: https://proceedings.mlr.press/v220/yip23a.html
PDF: https://proceedings.mlr.press/v220/yip23a/yip23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-yip23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Kai Hou
family: Yip
- given: Quentin
family: Changeat
- given: Ingo
family: Waldmann
- given: Eyup B.
family: Unlu
- given: Roy T.
family: Forestano
- given: Alexander
family: Roman
- given: Katia
family: Matcheva
- given: Konstantin T.
family: Matchev
- given: Stefan
family: Stefanov
- given: Ond\vrej
family: Podsztavek
- given: Mario
family: Morvan
- given: Nikolaos
family: Nikolaou
- given: Ahmed
family: Al-Refaie
- given: Clare
family: Jenner
- given: Chris
family: Johnson
- given: Angelos
family: Tsiaras
- given: Billy
family: Edwards
- given: Catarina
prefix: Alves de
family: Oliveira
- given: Jeyan
family: Thiyagalingam
- given: Pierre-Olivier
family: Lagage
- given: James
family: Cho
- given: Giovanna
family: Tinetti
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 1-17
id: yip23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 1
lastpage: 17
published: 2023-08-31 00:00:00 +0000
- title: 'The NeurIPS 2022 Neural MMO Challenge: A Massively Multiagent Competition with Specialization and Trade'
abstract: 'In this paper, we present the results of the NeurIPS-2022 Neural MMO Challenge, which attracted 500 participants and received over 1,600 submissions. Like the previous IJCAI-2022 Neural MMO Challenge, it involved agents from 16 populations surviving in procedurally generated worlds by collecting resources and defeating opponents. This year’s competition runs on the latest v1.6 Neural MMO, which introduces new equipment, combat, trading, and a better scoring system. These elements combine to pose additional robustness and generalization challenges not present in previous competitions. This paper summarizes the design and results of the challenge, explores the potential of this environment as a benchmark for learning methods, and presents some practical reinforcement learning training approaches for complex tasks with sparse rewards. Additionally, we have open-sourced our baselines, including environment wrappers, benchmarks, and visualization tools for future research.'
volume: 220
URL: https://proceedings.mlr.press/v220/liu23a.html
PDF: https://proceedings.mlr.press/v220/liu23a/liu23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-liu23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Enhong
family: Liu
- given: Joseph
family: Suarez
- given: Chenhui
family: You
- given: Bo
family: Wu
- given: Bingcheng
family: Chen
- given: Jun
family: Hu
- given: Jiaxin
family: Chen
- given: Xiaolong
family: Zhu
- given: Clare
family: Zhu
- given: Julian
family: Togelius
- given: Sharada
family: Mohanty
- given: Weijun
family: Hong
- given: Rui
family: Du
- given: Yibing
family: Zhang
- given: Qinwen
family: Wang
- given: Xinhang
family: Li
- given: Zheng
family: Yuan
- given: Xiang
family: Li
- given: Yuejia
family: Huang
- given: Kun
family: Zhang
- given: Hanhui
family: Yang
- given: Shiqi
family: Tang
- given: Phillip
family: Isola
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 18-34
id: liu23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 18
lastpage: 34
published: 2023-08-31 00:00:00 +0000
- title: 'The EURO Meets NeurIPS 2022 Vehicle Routing Competition'
abstract: 'Solving vehicle routing problems (VRPs) is an essential task for many industrial applications. Although VRPs have been traditionally studied in the operations research (OR) domain, they have lately been the subject of extensive work in the machine learning (ML) community. Both the OR and ML communities have begun to integrate ML into their methods, but in vastly different ways. While the OR community primarily relies on simplistic ML methods, the ML community generally uses deep learning, but fails to outperform OR baselines. To address this gap, the *EURO Meets NeurIPS 2022 Vehicle Routing Competition* brought together the OR and ML communities as a joint effort of several previous competitions to solve a challenging VRP variant on real-world data provided by ORTEC, a leading provider of vehicle routing software. The challenge focuses on both a "classic" deterministic VRP with time windows (VRPTW) and a dynamic version in which new orders arrive over the course of a day. Over 50 teams submitted solutions over a 13-week submission period, battling for not only the best performance on the competition problems, but also for the longest dominance of the leaderboard. The goals of the competition were achieved, with both state-of-the-art techniques in OR and ML playing a significant role in several of the winning submissions.'
volume: 220
URL: https://proceedings.mlr.press/v220/kool23a.html
PDF: https://proceedings.mlr.press/v220/kool23a/kool23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-kool23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Wouter
family: Kool
- given: Laurens
family: Bliek
- given: Danilo
family: Numeroso
- given: Yingqian
family: Zhang
- given: Tom
family: Catshoek
- given: Kevin
family: Tierney
- given: Thibaut
family: Vidal
- given: Joaquim
family: Gromicho
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 35-49
id: kool23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 35
lastpage: 49
published: 2023-08-31 00:00:00 +0000
- title: 'NeurIPS’22 Cross-Domain MetaDL Challenge: Results and lessons learned'
abstract: 'Deep neural networks have demonstrated the ability to outperform humans in multiple tasks, but they often require substantial amounts of data and computational resources. These resources may be limited in certain fields. Meta-learning seeks to overcome these challenges by utilizing past task experiences to efficiently solve new tasks, achieving better performance with limited training data and modest computational resources. To further advance the ChaLearn MetaDL competition series, we organized the Cross-Domain MetaDL Challenge for NeurIPS’22. This challenge aimed to solve “any-way" and “any-shot" tasks from 10 domains through cross-domain meta-learning. In this paper, authored collaboratively by the competition organizers, top-ranked participants, and external collaborators, we describe the technical aspects of the competition, baseline methods, and top-ranked approaches that have been open-sourced. Additionally, we provide a detailed analysis of the competition results. Lessons learned from this competition include the critical role of pre-trained backbones, the necessity of preventing overfitting, and the significance of using data augmentation or domain adaptation techniques in conjunction with extra optimizations to improve performance.'
volume: 220
URL: https://proceedings.mlr.press/v220/carrion-ojeda23a.html
PDF: https://proceedings.mlr.press/v220/carrion-ojeda23a/carrion-ojeda23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-carrion-ojeda23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Dustin
family: Carrión-Ojeda
- given: Mahbubul
family: Alam
- given: Sergio
family: Escalera
- given: Ahmed
family: Farahat
- given: Dipanjan
family: Ghosh
- given: Teresa
family: Gonzalez Diaz
- given: Chetan
family: Gupta
- given: Isabelle
family: Guyon
- given: Joël Roman
family: Ky
- given: Xian Yeow
family: Lee
- given: Xin
family: Liu
- given: Felix
family: Mohr
- given: Manh Hung
family: Nguyen
- given: Emmanuel
family: Pintelas
- given: Stefan
family: Roth
- given: Simone
family: Schaub-Meyer
- given: Haozhe
family: Sun
- given: Ihsan
family: Ullah
- given: Joaquin
family: Vanschoren
- given: Lasitha
family: Vidyaratne
- given: Jiamin
family: Wu
- given: Xiaotian
family: Yin
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 50-72
id: carrion-ojeda23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 50
lastpage: 72
published: 2023-08-31 00:00:00 +0000
- title: 'Driving SMARTS Competition at NeurIPS 2022: Insights and Outcome'
abstract: 'The Driving SMARTS (Scalable Multi-Agent Reinforcement Learning Training School) competition was designed to address one of the major challenges for autonomous driving (AD), namely adaptation to distribution shift between data used for training and inference and the problems caused by this shift in real-world conditions. The two key features of the competition are 1) a two-track structure to encourage and support a variety of approaches to solving the problem, such as reinforcement learning, offline learning, and other machine learning methods; and 2) curated data for driving scenarios of varying difficulty levels, from cruising to unprotected turns at unsignalized intersections. The competition attracted 87 participants in 53 teams. Top-ranking teams contributed a diverse set of solutions highlighting the effectiveness of different methodologies on safe motion planning for AD. This paper provides an overview of the Driving SMARTS competition, discusses its organisational and design aspects, and presents the results, insights, and promising directions for future research.'
volume: 220
URL: https://proceedings.mlr.press/v220/rasouli23a.html
PDF: https://proceedings.mlr.press/v220/rasouli23a/rasouli23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-rasouli23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Amir
family: Rasouli
- given: Soheil
family: Alizadeh
- given: Iuliia
family: Kotseruba
- given: Yi
family: Ma
- given: Hebin
family: Liang
- given: Yuan
family: Tian
- given: Zhiyu
family: Huang
- given: Haochen
family: Liu
- given: Jingda
family: Wu
- given: Randy
family: Goebel
- given: Tianpei
family: Yang
- given: Matthew E.
family: Taylor
- given: Liam
family: Paull
- given: Xi
family: Chen
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 73-84
id: rasouli23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 73
lastpage: 84
published: 2023-08-31 00:00:00 +0000
- title: 'The CityLearn Challenge 2022: Overview, Results, and Lessons Learned'
abstract: 'The shift to renewable power sources and building electrification to decarbonize existing and emerging building stock present unique challenges for the power grid. Building loads and flexible resources e.g. batteries must be adequately managed simultaneously to unlock the full flexibility potential and reduce costs for all stakeholders. Simple control algorithms based on expert knowledge e.g. RBC, as well as, advanced control algorithms e.g. MPC and RLC can be utilized to intelligently manage flexible resources. The CityLearn Challenge is an opportunity to compete in investigating the potential of AI and distributed control systems to tackle multiple problems within the built-environment. The CityLearn Challenge 2022 is the third of its kind with the overall objective of crowd-sourcing generalizable control policies that improve energy, cost and environmental objectives by taking advantage of batteries for load shifting in a CityLearn digital twin of a real-world grid-interactive neighborhood. Highlighted here are the uniqueness of this third edition, baseline and top solutions, and lessons learned for future editions.'
volume: 220
URL: https://proceedings.mlr.press/v220/nweye23a.html
PDF: https://proceedings.mlr.press/v220/nweye23a/nweye23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-nweye23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Kingsley
family: Nweye
- given: Zoltan
family: Nagy
- given: Sharada
family: Mohanty
- given: Dipam
family: Chakraborty
- given: Siva
family: Sankaranarayanan
- given: Tianzhen
family: Hong
- given: Sourav
family: Dey
- given: Gregor
family: Henze
- given: Jan
family: Drgona
- given: Fangquan
family: Lin
- given: Wei
family: Jiang
- given: Hanwei
family: Zhang
- given: Zhongkai
family: Yi
- given: Jihai
family: Zhang
- given: Cheng
family: Yang
- given: Matthew
family: Motoki
- given: Sorapong
family: Khongnawang
- given: Michael
family: Ibrahim
- given: Abilmansur
family: Zhumabekov
- given: Daniel
family: May
- given: Zhihu
family: Yang
- given: Xiaozhuang
family: Song
- given: Han
family: Zhang
- given: Xiaoning
family: Dong
- given: Shun
family: Zheng
- given: Jiang
family: Bian
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 85-103
id: nweye23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 85
lastpage: 103
published: 2023-08-31 00:00:00 +0000
- title: 'VisDA 2022 Challenge: Domain Adaptation for Industrial Waste Sorting'
abstract: 'Label-efficient and reliable semantic segmentation is essential for many real-life applications, especially for industrial settings with high visual diversity, such as waste sorting. In industrial waste sorting, one of the biggest challenges is the extreme diversity of the input stream depending on factors like the location of the sorting facility, the equipment available in the facility, and the time of year, all of which significantly impact the composition and visual appearance of the waste stream. These changes in the data are called “visual domains”, and label-efficient adaptation of models to such domains is needed for successful semantic segmentation of industrial waste. To test the abilities of computer vision models on this task, we present the \emph{VisDA 2022 Challenge on Domain Adaptation for Industrial Waste Sorting}. Our challenge incorporates a fully-annotated waste sorting dataset, ZeroWaste, collected from two real material recovery facilities in different locations and seasons, as well as a novel procedurally generated synthetic waste sorting dataset, SynthWaste. In this competition, we aim to answer two questions: 1) can we leverage domain adaptation techniques to minimize the domain gap? and 2) can synthetic data augmentation improve performance on this task and help adapt to changing data distributions? The results of the competition show that industrial waste detection poses a real domain adaptation problem, that domain generalization techniques such as augmentations, ensembling, etc., improve the overall performance on the unlabeled target domain examples, and that leveraging synthetic data effectively remains an open problem. See \url{https://ai.bu.edu/visda-2022/}'
volume: 220
URL: https://proceedings.mlr.press/v220/bashkirova23a.html
PDF: https://proceedings.mlr.press/v220/bashkirova23a/bashkirova23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-bashkirova23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Dina
family: Bashkirova
- given: Samarth
family: Mishra
- given: Diala
family: Lteif
- given: Piotr
family: Teterwak
- given: Donghyun
family: Kim
- given: Fadi
family: Alladkani
- given: James
family: Akl
- given: Berk
family: Calli
- given: Sarah Adel
family: Bargal
- given: Kate
family: Saenko
- given: Daehan
family: Kim
- given: Minseok
family: Seo
- given: YoungJin
family: Jeon
- given: Dong-Geol
family: Choi
- given: Shahaf
family: Ettedgui
- given: Raja
family: Giryes
- given: Shady
family: Abu-Hussein
- given: Binhui
family: Xie
- given: Shuang
family: Li
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 104-118
id: bashkirova23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 104
lastpage: 118
published: 2023-08-31 00:00:00 +0000
- title: 'The Machine Reconnaissance Blind Chess Tournament of NeurIPS 2022'
abstract: 'Reconnaissance Blind Chess is a game that plays like regular chess but rather than continuously observing the entire board, each player can only momentarily and privately observe selected board regions. It has imperfect information and little common knowledge. The Johns Hopkins University Applied Physics Laboratory (the game’s creator) and several partners organized the third NeurIPS machine Reconnaissance Blind Chess competition in 2022 to bring people together to attempt to tackle research challenges presented by the game. 18 bots played each other in 9,180 games (60 matches per bot pair) over 4 days. The top bot exceeded the performance of all of last year’s bots yet a practical, sound (unexploitable) algorithm remains unknown.'
volume: 220
URL: https://proceedings.mlr.press/v220/gardner23a.html
PDF: https://proceedings.mlr.press/v220/gardner23a/gardner23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-gardner23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Ryan W.
family: Gardner
- given: Gino
family: Perrotta
- given: Anvay
family: Shah
- given: Shivaram
family: Kalyanakrishnan
- given: Kevin A.
family: Wang
- given: Gregory
family: Clark
- given: Timo
family: Bertram
- given: Johannes
family: Fürnkranz
- given: Martin
family: Müller
- given: Brady P.
family: Garrison
- given: Prithviraj
family: Dasgupta
- given: Saeid
family: Rezaei
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 119-132
id: gardner23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 119
lastpage: 132
published: 2023-08-31 00:00:00 +0000
- title: 'Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real World'
abstract: 'Experimentation on real robots is demanding in terms of time and costs. For this reason, a large part of the reinforcement learning (RL) community uses simulators to develop and benchmark algorithms. However, insights gained in simulation do not necessarily translate to real robots, in particular for tasks involving complex interactions with the environment. The *Real Robot Challenge 2022* therefore served as a bridge between the RL and robotics communities by allowing participants to experiment remotely with a *real* robot - as easily as in simulation. In the last years, offline reinforcement learning has matured into a promising paradigm for learning from pre-collected datasets, alleviating the reliance on expensive online interactions. We therefore asked the participants to learn two dexterous manipulation tasks involving pushing, grasping, and in-hand orientation from provided real-robot datasets. An extensive software documentation and an initial stage based on a simulation of the real set-up made the competition particularly accessible. By giving each team plenty of access budget to evaluate their offline-learned policies on a cluster of seven identical real TriFinger platforms, we organized an exciting competition for machine learners and roboticists alike. In this work we state the rules of the competition, present the methods used by the winning teams and compare their results with a benchmark of state-of-the-art offline RL algorithms on the challenge datasets.'
volume: 220
URL: https://proceedings.mlr.press/v220/gurtler23a.html
PDF: https://proceedings.mlr.press/v220/gurtler23a/gurtler23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-gurtler23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Nico
family: Gürtler
- given: Felix
family: Widmaier
- given: Cansu
family: Sancaktar
- given: Sebastian
family: Blaes
- given: Pavel
family: Kolev
- given: Stefan
family: Bauer
- given: Manuel
family: Wüthrich
- given: Markus
family: Wulfmeier
- given: Martin
family: Riedmiller
- given: Arthur
family: Allshire
- given: Qiang
family: Wang
- given: Robert
family: McCarthy
- given: Hangyeol
family: Kim
- given: Jongchan
family: Baek
- given: Wookyong
family: Kwon
- given: Shanliang
family: Qian
- given: Yasunori
family: Toshimitsu
- given: Mike Yan
family: Michelis
- given: Amirhossein
family: Kazemipour
- given: Arman
family: Raayatsanati
- given: Hehui
family: Zheng
- given: Barnabas Gavin
family: Cangan
- given: Bernhard
family: Schölkopf
- given: Georg
family: Martius
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 133-150
id: gurtler23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 133
lastpage: 150
published: 2023-08-31 00:00:00 +0000
- title: 'AutoML Decathlon: Diverse Tasks, Modern Methods, and Efficiency at Scale'
abstract: 'The vision of Automated Machine Learning (AutoML) is to produce high performing ML pipelines that require very little human involvement or domain expertise to use. Competitions and benchmarks have been critical tools for accelerating progress in AutoML. However, much of the prior work on AutoML competitions has focused on well-studied domains in machine learning such as vision and language—these are domains which have benefited from several years of ML pipeline design by domain experts, which brings the usage of AutoML into question in the first place. Recently, AutoML for diverse tasks has emerged as an important research area that aims to bring AutoML to the domains where it can have the most impact: the long tail of ML tasks *beyond vision and language*. We present a retrospective report of the AutoML Decathlon—an AutoML for diverse tasks competition hosted at NeurIPS 2022. The AutoML Decathlon presented participants with a set of 10 machine learning tasks that are diverse along several axes: domain, input dimension, output dimension, output type, objective function, and scale. Participants were tasked with developing AutoML methods that performed well on a *separate* set of 10 hidden diverse test tasks within a certain time budget, so as to discourage overfitting to the initial set of tasks and to encourage efficiency. In this report, we outline the details of the competition, discuss the top-5 submissions, analyze the results, and compare top submissions to additional state-of-the-art baselines designed specifically for diverse tasks. We conclude that the combination of existing efficient AutoML techniques with modern advancements in ML such as large-scale transfer learning, modern architectures, and differentiable Neural Architecture Search (NAS) is a promising direction for AutoML for diverse tasks.'
volume: 220
URL: https://proceedings.mlr.press/v220/roberts23a.html
PDF: https://proceedings.mlr.press/v220/roberts23a/roberts23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-roberts23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Nicholas
family: Roberts
- given: Samuel
family: Guo
- given: Cong
family: Xu
- given: Ameet
family: Talwalkar
- given: David
family: Lander
- given: Lvfang
family: Tao
- given: Linhang
family: Cai
- given: Shuaicheng
family: Niu
- given: Jianyu
family: Heng
- given: Hongyang
family: Qin
- given: Minwen
family: Deng
- given: Johannes
family: Hog
- given: Alexander
family: Pfefferle
- given: Sushil Ammanaghatta
family: Shivakumar
- given: Arjun
family: Krishnakumar
- given: Yubo
family: Wang
- given: Rhea
family: Sukthanker
- given: Frank
family: Hutter
- given: Euxhen
family: Hasanaj
- given: Tien-Dung
family: Le
- given: Mikhail
family: Khodak
- given: Yuriy
family: Nevmyvaka
- given: Kashif
family: Rasul
- given: Frederic
family: Sala
- given: Anderson
family: Schneider
- given: Junhong
family: Shen
- given: Evan
family: Sparks
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 151-170
id: roberts23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 151
lastpage: 170
published: 2023-08-31 00:00:00 +0000
- title: 'Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition'
abstract: 'To facilitate research in the direction of fine-tuning foundation models from human feedback, we held the MineRL BASALT Competition on Fine-Tuning from Human Feedback at NeurIPS 2022. The BASALT challenge asks teams to compete to develop algorithms to solve tasks with hard-to-specify reward functions in Minecraft. Through this competition, we aimed to promote the development of algorithms that use human feedback as channels to learn the desired behavior. We describe the competition and provide an overview of the top solutions. We conclude by discussing the impact of the competition and future directions for improvement.'
volume: 220
URL: https://proceedings.mlr.press/v220/milani23a.html
PDF: https://proceedings.mlr.press/v220/milani23a/milani23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-milani23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Stephanie
family: Milani
- given: Anssi
family: Kanervisto
- given: Karolis
family: Ramanauskas
- given: Sander
family: Schulhoff
- given: Brandon
family: Houghton
- given: Sharada
family: Mohanty
- given: Byron
family: Galbraith
- given: Ke
family: Chen
- given: Yan
family: Song
- given: Tianze
family: Zhou
- given: Bingquan
family: Yu
- given: He
family: Liu
- given: Kai
family: Guan
- given: Yujing
family: Hu
- given: Tangjie
family: Lv
- given: Federico
family: Malato
- given: Florian
family: Leopold
- given: Amogh
family: Raut
- given: Ville
family: Hautamäki
- given: Andrew
family: Melnik
- given: Shu
family: Ishida
- given: João
family: Henriques
- given: Robert
family: Klassert
- given: Walter
family: Laurito
- given: Lucas
family: Cazzonelli
- given: Cedric
family: Kulbach
- given: Nicholas
family: Popovic
- given: Marvin
family: Schweizer
- given: Ellen
family: Novoseller
- given: Vinicius
family: Goecks
- given: Nicholas
family: Waytowich
- given: David
family: Watkins
- given: Josh
family: Miller
- given: Rohin
family: Shah
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 171-188
id: milani23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 171
lastpage: 188
published: 2023-08-31 00:00:00 +0000
- title: 'NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions'
abstract: 'The Natural Language for Optimization (NL4Opt) Competition was created to investigate methods of extracting the meaning and formulation of an optimization problem based on its text description. Specifically, the goal of the competition is to increase the accessibility and usability of optimization solvers by allowing non-experts to interface with them using natural language. We separate this challenging goal into two sub-tasks: (1) recognize and label the semantic entities that correspond to the components of the optimization problem; (2) generate a meaning representation (i.e. a logical form) of the problem from its detected problem entities. The first task aims to reduce ambiguity by detecting and tagging the entities of the optimization problems. The second task creates an intermediate representation of the linear programming (LP) problem that is converted into a format that can be used by commercial solvers. In this report, we present the LP word problem dataset and shared tasks for the NeurIPS 2022 competition. Furthermore, we present the winning solutions. Through this competition, we hope to bring interest towards the development of novel machine learning applications and datasets for optimization modeling.'
volume: 220
URL: https://proceedings.mlr.press/v220/ramamonjison23a.html
PDF: https://proceedings.mlr.press/v220/ramamonjison23a/ramamonjison23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-ramamonjison23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Rindranirina
family: Ramamonjison
- given: Timothy
family: Yu
- given: Raymond
family: Li
- given: Haley
family: Li
- given: Giuseppe
family: Carenini
- given: Bissan
family: Ghaddar
- given: Shiqi
family: He
- given: Mahdi
family: Mostajabdaveh
- given: Amin
family: Banitalebi-Dehkordi
- given: Zirui
family: Zhou
- given: Yong
family: Zhang
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 189-203
id: ramamonjison23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 189
lastpage: 203
published: 2023-08-31 00:00:00 +0000
- title: 'Interactive Grounded Language Understanding in a Collaborative Environment: Retrospective on Iglu 2022 Competition'
abstract: 'Human intelligence possesses the extraordinary ability to adapt rapidly to new tasks and multi-modal environments. This capacity emerges at an early age, as humans acquire new skills and learn to solve problems by imitating others or following natural language instructions. To facilitate research in this area, we recently hosted the second \emph{IGLU: Interactive Grounded Language Understanding in a Collaborative Environment} competition. The primary objective of the competition is to address the challenge of creating interactive agents that can learn to solve complex tasks by receiving grounded natural language instructions in a collaborative environment. Given the complexity of this challenge, we divided it into two sub-tasks: first, deciding whether the provided grounded instruction requires clarification, and second, following a clear grounded instruction to complete the task description.'
volume: 220
URL: https://proceedings.mlr.press/v220/kiseleva23a.html
PDF: https://proceedings.mlr.press/v220/kiseleva23a/kiseleva23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-kiseleva23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Julia
family: Kiseleva
- given: Alexey
family: Skrynnik
- given: Artem
family: Zholus
- given: Shrestha
family: Mohanty
- given: Negar
family: Arabzadeh
- given: Marc-Alexandre
family: Côté
- given: Mohammad
family: Aliannejadi
- given: Milagro
family: Teruel
- given: Ziming
family: Li
- given: Mikhail
family: Burtsev
- given: Maartje
prefix: ter
family: Hoeve
- given: Zoya
family: Volovikova
- given: Aleksandr
family: Panov
- given: Yuxuan
family: Sun
- given: Kavya
family: Srinet
- given: Arthur
family: Szlam
- given: Ahmed
family: Awadallah
- given: Seungeun
family: Rho
- given: Taehwan
family: Kwon
- given: Daniel
family: Wontae Nam
- given: Felipe
family: Bivort Haiek
- given: Edwin
family: Zhang
- given: Linar
family: Abdrazakov
- given: Guo
family: Qingyam
- given: Jason
family: Zhang
- given: Zhibin
family: Guo
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 204-216
id: kiseleva23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 204
lastpage: 216
published: 2023-08-31 00:00:00 +0000
- title: 'Findings of the Second AmericasNLP Competition on Speech-to-Text Translation'
abstract: 'Indigenous languages, including those from the Americas, have received very little attention from the machine learning (ML) and natural language processing (NLP) communities. To tackle the resulting lack of systems for these languages and the accompanying social inequalities affecting their speakers, we conduct the second AmericasNLP competition (and the first one in collaboration with NeurIPS), which is centered around speech-to-text translation systems for Indigenous languages of the Americas. The competition features three tasks – (1) automatic speech recognition, (2) text-based machine translation, and (3) speech-to-text translation – and two tracks: constrained and unconstrained. Five Indigenous languages are covered: Bribri, Guarani, Kotiria, Wa’ikhana, and Quechua. In this overview paper, we describe the tasks, tracks, and languages, introduce the baseline and participating systems, and end with a summary of ongoing and future challenges for the automatic translation of Indigenous languages.'
volume: 220
URL: https://proceedings.mlr.press/v220/ebrahimi23a.html
PDF: https://proceedings.mlr.press/v220/ebrahimi23a/ebrahimi23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-ebrahimi23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Abteen
family: Ebrahimi
- given: Manuel
family: Mager
- given: Adam
family: Wiemerslage
- given: Pavel
family: Denisov
- given: Arturo
family: Oncevay
- given: Danni
family: Liu
- given: Sai
family: Koneru
- given: Enes Yavuz
family: Ugan
- given: Zhaolin
family: Li
- given: Jan
family: Niehues
- given: Monica
family: Romero
- given: Ivan G
family: Torre
- given: Tanel
family: Alumäe
- given: Jiaming
family: Kong
- given: Sergey
family: Polezhaev
- given: Yury
family: Belousov
- given: Wei-Rui
family: Chen
- given: Peter
family: Sullivan
- given: Ife
family: Adebara
- given: Bashar
family: Talafha
- given: Alcides Alcoba
family: Inciarte
- given: Muhammad
family: Abdul-Mageed
- given: Luis
family: Chiruzzo
- given: Rolando
family: Coto-Solano
- given: Hilaria
family: Cruz
- given: Sofía
family: Flores-Solórzano
- given: Aldo Andrés Alvarez
family: López
- given: Ivan
family: Meza-Ruiz
- given: John E.
family: Ortega
- given: Alexis
family: Palmer
- given: Rodolfo Joel Zevallos
family: Salazar
- given: Kristine
family: Stenzel
- given: Thang
family: Vu
- given: Katharina
family: Kann
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 217-232
id: ebrahimi23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 217
lastpage: 232
published: 2023-08-31 00:00:00 +0000
- title: 'MyoChallenge 2022: Learning contact-rich manipulation using a musculoskeletal hand'
abstract: 'Manual dexterity has been considered one of the critical components for human evolution. The ability to perform movements as simple as holding and rotating an object in the hand without dropping it needs the coordination of more than 35 muscles which act synergistically or antagonistically on multiple joints. This complexity in control is markedly different from typical pre-specified movements or torque based controls used in robotics. In the MyoChallenge at the NeurIPS 2022 competition track, we challenged the community to develop controllers for a realistic hand to solve a series of dexterous manipulation tasks. The MyoSuite framework was used to train and test controllers on realistic, contact rich and computation efficient virtual neuromusculoskeletal model of the hand and wrist. Two tasks were proposed: a die re-orientation and a boading ball (rotation of two spheres respect to each other) tasks. More than 40 teams participated to the challenge and submitted more than 340 solutions. The challenge was split in two phases. In the first phase, where a limited set of objectives and randomization were proposed, teams managed to achieve high performance, in particular in the boading-ball task. In the second phase as the focus shifted towards generalization of task solutions to extensive variations of object and task properties, teams saw significant performance drop. This shows that there is still a large gap in developing agents capable of generalizable skilled manipulation. In future challenges, we will continue pursuing the generalizability both in skills and agility of the tasks exploring additional realistic neuromusculoskeletal models.'
volume: 220
URL: https://proceedings.mlr.press/v220/caggiano23a.html
PDF: https://proceedings.mlr.press/v220/caggiano23a/caggiano23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-caggiano23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Vittorio
family: Caggiano
- given: Guillaume
family: Durandau
- given: Huwawei
family: Wang
- given: Alberto
family: Chiappa
- given: Alexander
family: Mathis
- given: Pablo
family: Tano
- given: Nisheet
family: Patel
- given: Alexandre
family: Pouget
- given: Pierre
family: Schumacher
- given: Georg
family: Martius
- given: Daniel
family: Haeufle
- given: Yiran
family: Geng
- given: Boshi
family: An
- given: Yifan
family: Zhong
- given: Jiaming
family: Ji
- given: Yuanpei
family: Chen
- given: Hao
family: Dong
- given: Yaodong
family: Yang
- given: Rahul
family: Siripurapu
- given: Luis Eduardo
family: Ferro Diez
- given: Michael
family: Kopp
- given: Vihang
family: Patil
- given: Sepp
family: Hochreiter
- given: Yuval
family: Tassa
- given: Josh
family: Merel
- given: Randy
family: Schultheis
- given: Seungmoon
family: Song
- given: Massimo
family: Sartori
- given: Vikash
family: Kumar
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 233-250
id: caggiano23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 233
lastpage: 250
published: 2023-08-31 00:00:00 +0000
- title: 'Traffic4cast at NeurIPS 2022 – Predict Dynamics along Graph Edges from Sparse Node Data: Whole City Traffic and ETA from Stationary Vehicle Detectors'
abstract: 'The global trends of urbanization and increased personal mobility force us to rethink the way we live and use urban space. The Traffic4cast competition series tackles this problem in a data-driven way, advancing the latest methods in machine learning for modeling complex spatial systems over time. In this edition, our dynamic road graph data combine information from road maps, $10^{12}$ probe data points, and stationary vehicle detectors in three cities over the span of two years. While stationary vehicle detectors are the most accurate way to capture traffic volume, they are only available in few locations. Traffic4cast 2022 explores models that have the ability to generalize loosely related temporal vertex data on just a few nodes to predict dynamic future traffic states on the edges of the entire road graph. In the core challenge, participants are invited to predict the likelihoods of three congestion classes derived from the speed levels in the GPS data for the entire road graph in three cities 15 min into the future. We only provide vehicle count data from spatially sparse stationary vehicle detectors in these three cities as model input for this task. The data are aggregated in 15 min time bins for one hour prior to the prediction time. For the extended challenge, participants are tasked to predict the average travel times on super-segments 15 min into the future – super-segments are longer sequences of road segments in the graph. The competition results provide an important advance in the prediction of complex city-wide traffic states just from publicly available sparse vehicle data and without the need for large amounts of real-time floating vehicle data.'
volume: 220
URL: https://proceedings.mlr.press/v220/neun23a.html
PDF: https://proceedings.mlr.press/v220/neun23a/neun23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-neun23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Moritz
family: Neun
- given: Christian
family: Eichenberger
- given: Henry
family: Martin
- given: Markus
family: Spanring
- given: Rahul
family: Siripurapu
- given: Daniel
family: Springer
- given: Leyan
family: Deng
- given: Chenwang
family: Wu
- given: Defu
family: Lian
- given: Min
family: Zhou
- given: Martin
family: Lumiste
- given: Andrei
family: Ilie
- given: Xinhua
family: Wu
- given: Cheng
family: Lyu
- given: Qing-Long
family: Lu
- given: Vishal
family: Mahajan
- given: Yichao
family: Lu
- given: Jiezhang
family: Li
- given: Junjun
family: Li
- given: Yue-Jiao
family: Gong
- given: Florian
family: Grötschla
- given: Joël
family: Mathys
- given: Ye
family: Wei
- given: He
family: Haitao
- given: Hui
family: Fang
- given: Kevin
family: Malm
- given: Fei
family: Tang
- given: Michael
family: Kopp
- given: David
family: Kreil
- given: Sepp
family: Hochreiter
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 251-278
id: neun23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 251
lastpage: 278
published: 2023-08-31 00:00:00 +0000
- title: 'The Trojan Detection Challenge'
abstract: 'Neural trojan attacks inject machine learning systems with hidden behavior that lies dormant until activated. In recent years, trojan detection has emerged as a promising avenue for defending against standard trojan attacks. However, there have been few investigations on trojans specifically designed to be difficult to detect. We organized the Trojan Detection Challenge to begin work on the important question of how to build more robust trojan detectors. This paper gives an overview of the competition and its results. Notably, participants greatly improved over strong baselines on trojan detection and reverse-engineering tasks, demonstrating the potential for proactively improving the robustness of trojan detectors. We hope the competition and its results will inspire further research in detecting hidden behavior in machine learning systems.'
volume: 220
URL: https://proceedings.mlr.press/v220/mazeika23a.html
PDF: https://proceedings.mlr.press/v220/mazeika23a/mazeika23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-mazeika23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Mantas
family: Mazeika
- given: Dan
family: Hendrycks
- given: Huichen
family: Li
- given: Xiaojun
family: Xu
- given: Sidney
family: Hough
- given: Andy
family: Zou
- given: Arezoo
family: Rajabi
- given: Qi
family: Yao
- given: Zihao
family: Wang
- given: Jian
family: Tian
- given: Yao
family: Tang
- given: Di
family: Tang
- given: Roman
family: Smirnov
- given: Pavel
family: Pleskov
- given: Nikita
family: Benkovich
- given: Dawn
family: Song
- given: Radha
family: Poovendran
- given: Bo
family: Li
- given: David.
family: Forsyth
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 279-291
id: mazeika23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 279
lastpage: 291
published: 2023-08-31 00:00:00 +0000
- title: 'Weather4cast at NeurIPS 2022: Super-Resolution Rain Movie Prediction under Spatio-temporal Shifts'
abstract: 'Weather4cast again advanced modern algorithms in AI and machine learning through a highly topical interdisciplinary competition challenge: The prediction of hi-res rain radar movies from multi-band satellite sensors, requiring data fusion, multi-channel video frame prediction, and super-resolution. Accurate predictions of rain events are becoming ever more critical, with climate change increasing the frequency of unexpected rainfall. The resulting models will have a particular impact where costly weather radar is not available. We here present highlights and insights emerging from the thirty teams participating from over a dozen countries. To extract relevant patterns, models were challenged by spatio-temporal shifts. Geometric data augmentation and test-time ensemble models with a suitable smoother loss helped this transfer learning. Even though, in ablation, static information like geographical location and elevation was not linked to performance, the general success of models incorporating physics in this competition suggests that approaches combining machine learning with application domain knowledge seem a promising avenue for future research. Weather4cast will continue to explore the powerful benchmark reference data set introduced here, advancing competition tasks to quantitative predictions, and exploring the effects of metric choice on model performance and qualitative prediction properties.'
volume: 220
URL: https://proceedings.mlr.press/v220/gruca23a.html
PDF: https://proceedings.mlr.press/v220/gruca23a/gruca23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-gruca23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Aleksandra
family: Gruca
- given: Federico
family: Serva
- given: Llorenç
family: Lliso
- given: Pilar
family: Rípodas
- given: Xavier
family: Calbet
- given: Pedro
family: Herruzo
- given: Jiřı́
family: Pihrt
- given: Rudolf
family: Raevskyi
- given: Petr
family: Šimánek
- given: Matej
family: Choma
- given: Yang
family: Li
- given: Haiyu
family: Dong
- given: Yury
family: Belousov
- given: Sergey
family: Polezhaev
- given: Brian
family: Pulfer
- given: Minseok
family: Seo
- given: Doyi
family: Kim
- given: Seungheon
family: Shin
- given: Eunbin
family: Kim
- given: Sewoong
family: Ahn
- given: Yeji
family: Choi
- given: Jinyoung
family: Park
- given: Minseok
family: Son
- given: Seungju
family: Cho
- given: Inyoung
family: Lee
- given: Changick
family: Kim
- given: Taehyeon
family: Kim
- given: Shinhwan
family: Kang
- given: Hyeonjeong
family: Shin
- given: Deukryeol
family: Yoon
- given: Seongha
family: Eom
- given: Kijung
family: Shin
- given: Se-Young
family: Yun
- given: Bertrand
family: Le Saux
- given: Michael K
family: Kopp
- given: Sepp
family: Hochreiter
- given: David P
family: Kreil
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 292-313
id: gruca23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 292
lastpage: 313
published: 2023-08-31 00:00:00 +0000
- title: 'Retrospective on the SENSORIUM 2022 competition'
abstract: 'The neural underpinning of the biological visual system is challenging to study experimentally, in particular as neuronal activity becomes increasingly nonlinear with respect to visual input. Artificial neural networks (ANNs) can serve a variety of goals for improving our understanding of this complex system, not only serving as predictive digital twins of sensory cortex for novel hypothesis generation in silico, but also incorporating bio-inspired architectural motifs to progressively bridge the gap between biological and machine vision. The mouse has recently emerged as a popular model system to study visual information processing, but no standardized large-scale benchmark to identify state-of-the-art models of the mouse visual system has been established. To fill this gap, we proposed the SENSORIUM benchmark competition. We collected a large-scale dataset from mouse primary visual cortex containing the responses of more than 28,000 neurons across seven mice stimulated with thousands of natural images, together with simultaneous behavioral measurements that include running speed, pupil dilation, and eye movements. The benchmark challenge ranked models based on predictive performance for neuronal responses on a held-out test set, and included two tracks for model input limited to either stimulus only (SENSORIUM) or stimulus plus behavior (SENSORIUM+). As a part of the NeurIPS 2022 competition track, we received 172 model submissions from 26 teams, with the winning teams improving our previous state-of-the-art model by more than 15 percent. Dataset access and infrastructure for evaluation of model predictions will remain online as an ongoing benchmark. We would like to see this as a starting point for regular challenges and data releases, and as a standard tool for measuring progress in large-scale neural system identification models of the mouse visual system and beyond.'
volume: 220
URL: https://proceedings.mlr.press/v220/willeke23a.html
PDF: https://proceedings.mlr.press/v220/willeke23a/willeke23a.pdf
edit: https://github.com/mlresearch//v220/edit/gh-pages/_posts/2023-08-31-willeke23a.md
series: 'Proceedings of Machine Learning Research'
container-title: 'Proceedings of the NeurIPS 2022 Competitions Track'
publisher: 'PMLR'
author:
- given: Konstantin F.
family: Willeke
- given: Paul G.
family: Fahey
- given: Mohammad
family: Bashiri
- given: Laura
family: Hansel
- given: Christoph
family: Blessing
- given: Konstantin-Klemens
family: Lurz
- given: Max F.
family: Burg
- given: Santiago A.
family: Cadena
- given: Zhiwei
family: Ding
- given: Kayla
family: Ponder
- given: Taliah
family: Muhammad
- given: Saumil S.
family: Patel
- given: Kaiwen
family: Deng
- given: Yuanfang
family: Guan
- given: Yiqin
family: Zhu
- given: Kaiwen
family: Xiao
- given: Xiao
family: Han
- given: Simone
family: Azeglio
- given: Ulisse
family: Ferrari
- given: Peter
family: Neri
- given: Olivier
family: Marre
- given: Adrian
family: Hoffmann
- given: Kirill
family: Fedyanin
- given: Kirill
family: Vishniakov
- given: Maxim
family: Panov
- given: Subash
family: Prakash
- given: Kishan
family: Naik
- given: Kantharaju
family: Narayanappa
- given: Alexander S.
family: Ecker
- given: Andreas S.
family: Tolias
- given: Fabian H.
family: Sinz
editor:
- given: Marco
family: Ciccone
- given: Gustavo
family: Stolovitzky
- given: Jacob
family: Albrecht
page: 314-333
id: willeke23a
issued:
date-parts:
- 2022
- 8
- 31
firstpage: 314
lastpage: 333
published: 2023-08-31 00:00:00 +0000