Encoding Musical Style with Transformer Autoencoders

Kristy Choi, Curtis Hawthorne, Ian Simon, Monica Dinculescu, Jesse Engel
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:1899-1908, 2020.

Abstract

We consider the problem of learning high-level controls over the global structure of generated sequences, particularly in the context of symbolic music generation with complex language models. In this work, we present the Transformer autoencoder, which aggregates encodings of the input data across time to obtain a global representation of style from a given performance. We show it is possible to combine this global representation with other temporally distributed embeddings, enabling improved control over the separate aspects of performance style and melody. Empirically, we demonstrate the effectiveness of our method on various music generation tasks on the MAESTRO dataset and a YouTube dataset with 10,000+ hours of piano performances, where we achieve improvements in terms of log-likelihood and mean listening scores as compared to baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-choi20b, title = {Encoding Musical Style with Transformer Autoencoders}, author = {Choi, Kristy and Hawthorne, Curtis and Simon, Ian and Dinculescu, Monica and Engel, Jesse}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {1899--1908}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/choi20b/choi20b.pdf}, url = {https://proceedings.mlr.press/v119/choi20b.html}, abstract = {We consider the problem of learning high-level controls over the global structure of generated sequences, particularly in the context of symbolic music generation with complex language models. In this work, we present the Transformer autoencoder, which aggregates encodings of the input data across time to obtain a global representation of style from a given performance. We show it is possible to combine this global representation with other temporally distributed embeddings, enabling improved control over the separate aspects of performance style and melody. Empirically, we demonstrate the effectiveness of our method on various music generation tasks on the MAESTRO dataset and a YouTube dataset with 10,000+ hours of piano performances, where we achieve improvements in terms of log-likelihood and mean listening scores as compared to baselines.} }
Endnote
%0 Conference Paper %T Encoding Musical Style with Transformer Autoencoders %A Kristy Choi %A Curtis Hawthorne %A Ian Simon %A Monica Dinculescu %A Jesse Engel %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-choi20b %I PMLR %P 1899--1908 %U https://proceedings.mlr.press/v119/choi20b.html %V 119 %X We consider the problem of learning high-level controls over the global structure of generated sequences, particularly in the context of symbolic music generation with complex language models. In this work, we present the Transformer autoencoder, which aggregates encodings of the input data across time to obtain a global representation of style from a given performance. We show it is possible to combine this global representation with other temporally distributed embeddings, enabling improved control over the separate aspects of performance style and melody. Empirically, we demonstrate the effectiveness of our method on various music generation tasks on the MAESTRO dataset and a YouTube dataset with 10,000+ hours of piano performances, where we achieve improvements in terms of log-likelihood and mean listening scores as compared to baselines.
APA
Choi, K., Hawthorne, C., Simon, I., Dinculescu, M. & Engel, J.. (2020). Encoding Musical Style with Transformer Autoencoders. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:1899-1908 Available from https://proceedings.mlr.press/v119/choi20b.html.

Related Material