Attention-Enhanced Pointer Network for Summarization with Key Information

Li Qiming, Gao Liang
Proceedings of 2024 International Conference on Machine Learning and Intelligent Computing, PMLR 245:138-146, 2024.

Abstract

Addressing the limitations of mainstream generative text summarization models, such as poor semantic quality, inappropriate allocation of weights to key information, and constraints in extracting the semantic essence of textual content by existing natural language generation models, we propose an Attention-Augmented Pointer Generation Network (AUPT). This model utilizes TextRank technology to extract crucial information, combines positional encoding with an adaptive masking mechanism to enhance positional attention scores, emphasizing the importance of key information in the text’s semantics. Furthermore, by integrating the T5-Pegasus model with the pointer generation network, it effectively handles unknown vocabulary and replication issues, enabling more accurate and reliable semantic representations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v245-qiming24a, title = {Attention-Enhanced Pointer Network for Summarization with Key Information}, author = {Qiming, Li and Liang, Gao}, booktitle = {Proceedings of 2024 International Conference on Machine Learning and Intelligent Computing}, pages = {138--146}, year = {2024}, editor = {Nianyin, Zeng and Pachori, Ram Bilas}, volume = {245}, series = {Proceedings of Machine Learning Research}, month = {26--28 Apr}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v245/main/assets/qiming24a/qiming24a.pdf}, url = {https://proceedings.mlr.press/v245/qiming24a.html}, abstract = {Addressing the limitations of mainstream generative text summarization models, such as poor semantic quality, inappropriate allocation of weights to key information, and constraints in extracting the semantic essence of textual content by existing natural language generation models, we propose an Attention-Augmented Pointer Generation Network (AUPT). This model utilizes TextRank technology to extract crucial information, combines positional encoding with an adaptive masking mechanism to enhance positional attention scores, emphasizing the importance of key information in the text’s semantics. Furthermore, by integrating the T5-Pegasus model with the pointer generation network, it effectively handles unknown vocabulary and replication issues, enabling more accurate and reliable semantic representations. } }
Endnote
%0 Conference Paper %T Attention-Enhanced Pointer Network for Summarization with Key Information %A Li Qiming %A Gao Liang %B Proceedings of 2024 International Conference on Machine Learning and Intelligent Computing %C Proceedings of Machine Learning Research %D 2024 %E Zeng Nianyin %E Ram Bilas Pachori %F pmlr-v245-qiming24a %I PMLR %P 138--146 %U https://proceedings.mlr.press/v245/qiming24a.html %V 245 %X Addressing the limitations of mainstream generative text summarization models, such as poor semantic quality, inappropriate allocation of weights to key information, and constraints in extracting the semantic essence of textual content by existing natural language generation models, we propose an Attention-Augmented Pointer Generation Network (AUPT). This model utilizes TextRank technology to extract crucial information, combines positional encoding with an adaptive masking mechanism to enhance positional attention scores, emphasizing the importance of key information in the text’s semantics. Furthermore, by integrating the T5-Pegasus model with the pointer generation network, it effectively handles unknown vocabulary and replication issues, enabling more accurate and reliable semantic representations.
APA
Qiming, L. & Liang, G.. (2024). Attention-Enhanced Pointer Network for Summarization with Key Information. Proceedings of 2024 International Conference on Machine Learning and Intelligent Computing, in Proceedings of Machine Learning Research 245:138-146 Available from https://proceedings.mlr.press/v245/qiming24a.html.

Related Material