How recurrent networks implement contextual processing in sentiment analysis

Niru Maheswaranathan, David Sussillo
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:6608-6619, 2020.

Abstract

Neural networks have a remarkable capacity for contextual processing{—}using recent or nearby inputs to modify processing of current input. For example, in natural language, contextual processing is necessary to correctly interpret negation (e.g. phrases such as "not bad"). However, our ability to understand how networks process context is limited. Here, we propose general methods for reverse engineering recurrent neural networks (RNNs) to identify and elucidate contextual processing. We apply these methods to understand RNNs trained on sentiment classification. This analysis reveals inputs that induce contextual effects, quantifies the strength and timescale of these effects, and identifies sets of these inputs with similar properties. Additionally, we analyze contextual effects related to differential processing of the beginning and end of documents. Using the insights learned from the RNNs we improve baseline Bag-of-Words models with simple extensions that incorporate contextual modification, recovering greater than 90% of the RNN’s performance increase over the baseline. This work yields a new understanding of how RNNs process contextual information, and provides tools that should provide similar insight more broadly.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-maheswaranathan20a, title = {How recurrent networks implement contextual processing in sentiment analysis}, author = {Maheswaranathan, Niru and Sussillo, David}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {6608--6619}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/maheswaranathan20a/maheswaranathan20a.pdf}, url = {https://proceedings.mlr.press/v119/maheswaranathan20a.html}, abstract = {Neural networks have a remarkable capacity for contextual processing{—}using recent or nearby inputs to modify processing of current input. For example, in natural language, contextual processing is necessary to correctly interpret negation (e.g. phrases such as "not bad"). However, our ability to understand how networks process context is limited. Here, we propose general methods for reverse engineering recurrent neural networks (RNNs) to identify and elucidate contextual processing. We apply these methods to understand RNNs trained on sentiment classification. This analysis reveals inputs that induce contextual effects, quantifies the strength and timescale of these effects, and identifies sets of these inputs with similar properties. Additionally, we analyze contextual effects related to differential processing of the beginning and end of documents. Using the insights learned from the RNNs we improve baseline Bag-of-Words models with simple extensions that incorporate contextual modification, recovering greater than 90% of the RNN’s performance increase over the baseline. This work yields a new understanding of how RNNs process contextual information, and provides tools that should provide similar insight more broadly.} }
Endnote
%0 Conference Paper %T How recurrent networks implement contextual processing in sentiment analysis %A Niru Maheswaranathan %A David Sussillo %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-maheswaranathan20a %I PMLR %P 6608--6619 %U https://proceedings.mlr.press/v119/maheswaranathan20a.html %V 119 %X Neural networks have a remarkable capacity for contextual processing{—}using recent or nearby inputs to modify processing of current input. For example, in natural language, contextual processing is necessary to correctly interpret negation (e.g. phrases such as "not bad"). However, our ability to understand how networks process context is limited. Here, we propose general methods for reverse engineering recurrent neural networks (RNNs) to identify and elucidate contextual processing. We apply these methods to understand RNNs trained on sentiment classification. This analysis reveals inputs that induce contextual effects, quantifies the strength and timescale of these effects, and identifies sets of these inputs with similar properties. Additionally, we analyze contextual effects related to differential processing of the beginning and end of documents. Using the insights learned from the RNNs we improve baseline Bag-of-Words models with simple extensions that incorporate contextual modification, recovering greater than 90% of the RNN’s performance increase over the baseline. This work yields a new understanding of how RNNs process contextual information, and provides tools that should provide similar insight more broadly.
APA
Maheswaranathan, N. & Sussillo, D.. (2020). How recurrent networks implement contextual processing in sentiment analysis. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:6608-6619 Available from https://proceedings.mlr.press/v119/maheswaranathan20a.html.

Related Material