MeanSum: A Neural Model for Unsupervised Multi-Document Abstractive Summarization

Eric Chu, Peter Liu
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:1223-1232, 2019.

Abstract

Abstractive summarization has been studied using neural sequence transduction methods with datasets of large, paired document-summary examples. However, such datasets are rare and the models trained from them do not generalize to other domains. Recently, some progress has been made in learning sequence-to-sequence mappings with only unpaired examples. In our work, we consider the setting where there are only documents (product or business reviews) with no summaries provided, and propose an end-to-end, neural model architecture to perform unsupervised abstractive summarization. Our proposed model consists of an auto-encoder where the mean of the representations of the input reviews decodes to a reasonable summary-review. We consider variants of the proposed architecture and perform an ablation study to show the importance of specific components. We show through metrics and human evaluation that the generated summaries are highly abstractive, fluent, relevant, and representative of the average sentiment of the input reviews. Finally, we collect a ground-truth evaluation dataset and show that our model outperforms a strong extractive baseline.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-chu19b, title = {{M}ean{S}um: A Neural Model for Unsupervised Multi-Document Abstractive Summarization}, author = {Chu, Eric and Liu, Peter}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {1223--1232}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/chu19b/chu19b.pdf}, url = {https://proceedings.mlr.press/v97/chu19b.html}, abstract = {Abstractive summarization has been studied using neural sequence transduction methods with datasets of large, paired document-summary examples. However, such datasets are rare and the models trained from them do not generalize to other domains. Recently, some progress has been made in learning sequence-to-sequence mappings with only unpaired examples. In our work, we consider the setting where there are only documents (product or business reviews) with no summaries provided, and propose an end-to-end, neural model architecture to perform unsupervised abstractive summarization. Our proposed model consists of an auto-encoder where the mean of the representations of the input reviews decodes to a reasonable summary-review. We consider variants of the proposed architecture and perform an ablation study to show the importance of specific components. We show through metrics and human evaluation that the generated summaries are highly abstractive, fluent, relevant, and representative of the average sentiment of the input reviews. Finally, we collect a ground-truth evaluation dataset and show that our model outperforms a strong extractive baseline.} }
Endnote
%0 Conference Paper %T MeanSum: A Neural Model for Unsupervised Multi-Document Abstractive Summarization %A Eric Chu %A Peter Liu %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-chu19b %I PMLR %P 1223--1232 %U https://proceedings.mlr.press/v97/chu19b.html %V 97 %X Abstractive summarization has been studied using neural sequence transduction methods with datasets of large, paired document-summary examples. However, such datasets are rare and the models trained from them do not generalize to other domains. Recently, some progress has been made in learning sequence-to-sequence mappings with only unpaired examples. In our work, we consider the setting where there are only documents (product or business reviews) with no summaries provided, and propose an end-to-end, neural model architecture to perform unsupervised abstractive summarization. Our proposed model consists of an auto-encoder where the mean of the representations of the input reviews decodes to a reasonable summary-review. We consider variants of the proposed architecture and perform an ablation study to show the importance of specific components. We show through metrics and human evaluation that the generated summaries are highly abstractive, fluent, relevant, and representative of the average sentiment of the input reviews. Finally, we collect a ground-truth evaluation dataset and show that our model outperforms a strong extractive baseline.
APA
Chu, E. & Liu, P.. (2019). MeanSum: A Neural Model for Unsupervised Multi-Document Abstractive Summarization. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:1223-1232 Available from https://proceedings.mlr.press/v97/chu19b.html.

Related Material