Dance Dance Convolution

Chris Donahue, Zachary C. Lipton, Julian McAuley
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:1039-1048, 2017.

Abstract

Dance Dance Revolution (DDR) is a popular rhythm-based video game. Players perform steps on a dance platform in synchronization with music as directed by on-screen step charts. While many step charts are available in standardized packs, players may grow tired of existing charts, or wish to dance to a song for which no chart exists. We introduce the task of learning to choreograph. Given a raw audio track, the goal is to produce a new step chart. This task decomposes naturally into two subtasks: deciding when to place steps and deciding which steps to select. For the step placement task, we combine recurrent and convolutional neural networks to ingest spectrograms of low-level audio features to predict steps, conditioned on chart difficulty. For step selection, we present a conditional LSTM generative model that substantially outperforms n-gram and fixed-window approaches.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-donahue17a, title = {Dance Dance Convolution}, author = {Chris Donahue and Zachary C. Lipton and Julian McAuley}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {1039--1048}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/donahue17a/donahue17a.pdf}, url = {https://proceedings.mlr.press/v70/donahue17a.html}, abstract = {Dance Dance Revolution (DDR) is a popular rhythm-based video game. Players perform steps on a dance platform in synchronization with music as directed by on-screen step charts. While many step charts are available in standardized packs, players may grow tired of existing charts, or wish to dance to a song for which no chart exists. We introduce the task of learning to choreograph. Given a raw audio track, the goal is to produce a new step chart. This task decomposes naturally into two subtasks: deciding when to place steps and deciding which steps to select. For the step placement task, we combine recurrent and convolutional neural networks to ingest spectrograms of low-level audio features to predict steps, conditioned on chart difficulty. For step selection, we present a conditional LSTM generative model that substantially outperforms n-gram and fixed-window approaches.} }
Endnote
%0 Conference Paper %T Dance Dance Convolution %A Chris Donahue %A Zachary C. Lipton %A Julian McAuley %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-donahue17a %I PMLR %P 1039--1048 %U https://proceedings.mlr.press/v70/donahue17a.html %V 70 %X Dance Dance Revolution (DDR) is a popular rhythm-based video game. Players perform steps on a dance platform in synchronization with music as directed by on-screen step charts. While many step charts are available in standardized packs, players may grow tired of existing charts, or wish to dance to a song for which no chart exists. We introduce the task of learning to choreograph. Given a raw audio track, the goal is to produce a new step chart. This task decomposes naturally into two subtasks: deciding when to place steps and deciding which steps to select. For the step placement task, we combine recurrent and convolutional neural networks to ingest spectrograms of low-level audio features to predict steps, conditioned on chart difficulty. For step selection, we present a conditional LSTM generative model that substantially outperforms n-gram and fixed-window approaches.
APA
Donahue, C., Lipton, Z.C. & McAuley, J.. (2017). Dance Dance Convolution. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:1039-1048 Available from https://proceedings.mlr.press/v70/donahue17a.html.

Related Material