TAYSIR Competition: Transformer+\textscrnn: Algorithms to Yield Simple and Interpretable Representations

Rémi Eyraud, Dakotah Lambert, Badr Tahri Joutei, Aidar Gaffarov, Mathias Cabanne, Jeffrey Heinz, Chihiro Shibata
Proceedings of 16th edition of the International Conference on Grammatical Inference, PMLR 217:275-290, 2023.

Abstract

This article presents the content of the competition Transformers+\textsc{rnn}: Algorithms to Yield Simple and Interpretable Representations (TAYSIR, the Arabic word for ‘simple’), which was an on-line challenge on extracting simpler models from already trained neural networks held in Spring 2023. These neural nets were trained on sequential categorial/symbolic data. Some of these data were artificial, some came from real world problems (such as Natural Language Processing, Bioinformatics, and Software Engineering). The trained models covered a large spectrum of architectures, from Simple Recurrent Neural Network (SRN) to Transformers, including Gated Recurrent Unit (GRU) and Long Short Term Memory (LSTM). No constraint was given on the surrogate models submitted by the participants: any model working on sequential data was accepted. Two tracks were proposed: neural networks trained on Binary Classification tasks, and on Language Modeling tasks. The evaluation of the surrogate models took into account both the simplicity of the extracted model and the quality of the approximation of the original model.

Cite this Paper

BibTeX
@InProceedings{pmlr-v217-eyraud23a, title = {TAYSIR Competition: Transformer+\textsc{rnn}: Algorithms to Yield Simple and Interpretable Representations}, author = {Eyraud, R\'emi and Lambert, Dakotah and Tahri Joutei, Badr and Gaffarov, Aidar and Cabanne, Mathias and Heinz, Jeffrey and Shibata, Chihiro}, booktitle = {Proceedings of 16th edition of the International Conference on Grammatical Inference}, pages = {275--290}, year = {2023}, editor = {Coste, François and Ouardi, Faissal and Rabusseau, Guillaume}, volume = {217}, series = {Proceedings of Machine Learning Research}, month = {10--13 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v217/eyraud23a/eyraud23a.pdf}, url = {https://proceedings.mlr.press/v217/eyraud23a.html}, abstract = {This article presents the content of the competition Transformers+\textsc{rnn}: Algorithms to Yield Simple and Interpretable Representations (TAYSIR, the Arabic word for ‘simple’), which was an on-line challenge on extracting simpler models from already trained neural networks held in Spring 2023. These neural nets were trained on sequential categorial/symbolic data. Some of these data were artificial, some came from real world problems (such as Natural Language Processing, Bioinformatics, and Software Engineering). The trained models covered a large spectrum of architectures, from Simple Recurrent Neural Network (SRN) to Transformers, including Gated Recurrent Unit (GRU) and Long Short Term Memory (LSTM). No constraint was given on the surrogate models submitted by the participants: any model working on sequential data was accepted. Two tracks were proposed: neural networks trained on Binary Classification tasks, and on Language Modeling tasks. The evaluation of the surrogate models took into account both the simplicity of the extracted model and the quality of the approximation of the original model.} }
Endnote
%0 Conference Paper %T TAYSIR Competition: Transformer+\textscrnn: Algorithms to Yield Simple and Interpretable Representations %A Rémi Eyraud %A Dakotah Lambert %A Badr Tahri Joutei %A Aidar Gaffarov %A Mathias Cabanne %A Jeffrey Heinz %A Chihiro Shibata %B Proceedings of 16th edition of the International Conference on Grammatical Inference %C Proceedings of Machine Learning Research %D 2023 %E François Coste %E Faissal Ouardi %E Guillaume Rabusseau %F pmlr-v217-eyraud23a %I PMLR %P 275--290 %U https://proceedings.mlr.press/v217/eyraud23a.html %V 217 %X This article presents the content of the competition Transformers+\textsc{rnn}: Algorithms to Yield Simple and Interpretable Representations (TAYSIR, the Arabic word for ‘simple’), which was an on-line challenge on extracting simpler models from already trained neural networks held in Spring 2023. These neural nets were trained on sequential categorial/symbolic data. Some of these data were artificial, some came from real world problems (such as Natural Language Processing, Bioinformatics, and Software Engineering). The trained models covered a large spectrum of architectures, from Simple Recurrent Neural Network (SRN) to Transformers, including Gated Recurrent Unit (GRU) and Long Short Term Memory (LSTM). No constraint was given on the surrogate models submitted by the participants: any model working on sequential data was accepted. Two tracks were proposed: neural networks trained on Binary Classification tasks, and on Language Modeling tasks. The evaluation of the surrogate models took into account both the simplicity of the extracted model and the quality of the approximation of the original model.
APA
Eyraud, R., Lambert, D., Tahri Joutei, B., Gaffarov, A., Cabanne, M., Heinz, J. & Shibata, C.. (2023). TAYSIR Competition: Transformer+\textscrnn: Algorithms to Yield Simple and Interpretable Representations. Proceedings of 16th edition of the International Conference on Grammatical Inference, in Proceedings of Machine Learning Research 217:275-290 Available from https://proceedings.mlr.press/v217/eyraud23a.html.

Related Material