Explaining Black Boxes on Sequential Data using Weighted Automata

Stéphane Ayache, Rémi Eyraud, Noé Goudian
Proceedings of The 14th International Conference on Grammatical Inference 2018, PMLR 93:81-103, 2019.

Abstract

Understanding how a learned black box works is of crucial interest for the future of Machine Learning. In this paper, we pioneer the question of the global interpretability of learned black box models that assign numerical values to symbolic sequential data. To tackle that task, we propose a spectral algorithm for the extraction of weighted automata (WA) from such black boxes. This algorithm does not require the access to a dataset or to the inner representation of the black box: the inferred model can be obtained solely by querying the black box, feeding it with inputs and analyzing its outputs. Experiments using Recurrent Neural Networks (RNN) trained on a wide collection of 48 synthetic datasets and 2 real datasets show that the obtained approximation is of great quality.

Cite this Paper

BibTeX
@InProceedings{pmlr-v93-ayache19a, title = {Explaining Black Boxes on Sequential Data using Weighted Automata}, author = {Ayache, St\'ephane and Eyraud, R\'emi and Goudian, No\'e}, booktitle = {Proceedings of The 14th International Conference on Grammatical Inference 2018}, pages = {81--103}, year = {2019}, editor = {Unold, Olgierd and Dyrka, Witold and Wieczorek, Wojciech}, volume = {93}, series = {Proceedings of Machine Learning Research}, month = {feb}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v93/ayache19a/ayache19a.pdf}, url = {https://proceedings.mlr.press/v93/ayache19a.html}, abstract = {Understanding how a learned black box works is of crucial interest for the future of Machine Learning. In this paper, we pioneer the question of the global interpretability of learned black box models that assign numerical values to symbolic sequential data. To tackle that task, we propose a spectral algorithm for the extraction of weighted automata (WA) from such black boxes. This algorithm does not require the access to a dataset or to the inner representation of the black box: the inferred model can be obtained solely by querying the black box, feeding it with inputs and analyzing its outputs. Experiments using Recurrent Neural Networks (RNN) trained on a wide collection of 48 synthetic datasets and 2 real datasets show that the obtained approximation is of great quality.} }
Endnote
%0 Conference Paper %T Explaining Black Boxes on Sequential Data using Weighted Automata %A Stéphane Ayache %A Rémi Eyraud %A Noé Goudian %B Proceedings of The 14th International Conference on Grammatical Inference 2018 %C Proceedings of Machine Learning Research %D 2019 %E Olgierd Unold %E Witold Dyrka %E Wojciech Wieczorek %F pmlr-v93-ayache19a %I PMLR %P 81--103 %U https://proceedings.mlr.press/v93/ayache19a.html %V 93 %X Understanding how a learned black box works is of crucial interest for the future of Machine Learning. In this paper, we pioneer the question of the global interpretability of learned black box models that assign numerical values to symbolic sequential data. To tackle that task, we propose a spectral algorithm for the extraction of weighted automata (WA) from such black boxes. This algorithm does not require the access to a dataset or to the inner representation of the black box: the inferred model can be obtained solely by querying the black box, feeding it with inputs and analyzing its outputs. Experiments using Recurrent Neural Networks (RNN) trained on a wide collection of 48 synthetic datasets and 2 real datasets show that the obtained approximation is of great quality.
APA
Ayache, S., Eyraud, R. & Goudian, N.. (2019). Explaining Black Boxes on Sequential Data using Weighted Automata. Proceedings of The 14th International Conference on Grammatical Inference 2018, in Proceedings of Machine Learning Research 93:81-103 Available from https://proceedings.mlr.press/v93/ayache19a.html.

Related Material