Functional Transparency for Structured Data: a Game-Theoretic Approach

Guang-He Lee, Wengong Jin, David Alvarez-Melis, Tommi Jaakkola
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:3723-3733, 2019.

Abstract

We provide a new approach to training neural models to exhibit transparency in a well-defined, functional manner. Our approach naturally operates over structured data and tailors the predictor, functionally, towards a chosen family of (local) witnesses. The estimation problem is setup as a co-operative game between an unrestricted predictor such as a neural network, and a set of witnesses chosen from the desired transparent family. The goal of the witnesses is to highlight, locally, how well the predictor conforms to the chosen family of functions, while the predictor is trained to minimize the highlighted discrepancy. We emphasize that the predictor remains globally powerful as it is only encouraged to agree locally with locally adapted witnesses. We analyze the effect of the proposed approach, provide example formulations in the context of deep graph and sequence models, and empirically illustrate the idea in chemical property prediction, temporal modeling, and molecule representation learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-lee19b, title = {Functional Transparency for Structured Data: a Game-Theoretic Approach}, author = {Lee, Guang-He and Jin, Wengong and Alvarez-Melis, David and Jaakkola, Tommi}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {3723--3733}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/lee19b/lee19b.pdf}, url = {https://proceedings.mlr.press/v97/lee19b.html}, abstract = {We provide a new approach to training neural models to exhibit transparency in a well-defined, functional manner. Our approach naturally operates over structured data and tailors the predictor, functionally, towards a chosen family of (local) witnesses. The estimation problem is setup as a co-operative game between an unrestricted predictor such as a neural network, and a set of witnesses chosen from the desired transparent family. The goal of the witnesses is to highlight, locally, how well the predictor conforms to the chosen family of functions, while the predictor is trained to minimize the highlighted discrepancy. We emphasize that the predictor remains globally powerful as it is only encouraged to agree locally with locally adapted witnesses. We analyze the effect of the proposed approach, provide example formulations in the context of deep graph and sequence models, and empirically illustrate the idea in chemical property prediction, temporal modeling, and molecule representation learning.} }
Endnote
%0 Conference Paper %T Functional Transparency for Structured Data: a Game-Theoretic Approach %A Guang-He Lee %A Wengong Jin %A David Alvarez-Melis %A Tommi Jaakkola %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-lee19b %I PMLR %P 3723--3733 %U https://proceedings.mlr.press/v97/lee19b.html %V 97 %X We provide a new approach to training neural models to exhibit transparency in a well-defined, functional manner. Our approach naturally operates over structured data and tailors the predictor, functionally, towards a chosen family of (local) witnesses. The estimation problem is setup as a co-operative game between an unrestricted predictor such as a neural network, and a set of witnesses chosen from the desired transparent family. The goal of the witnesses is to highlight, locally, how well the predictor conforms to the chosen family of functions, while the predictor is trained to minimize the highlighted discrepancy. We emphasize that the predictor remains globally powerful as it is only encouraged to agree locally with locally adapted witnesses. We analyze the effect of the proposed approach, provide example formulations in the context of deep graph and sequence models, and empirically illustrate the idea in chemical property prediction, temporal modeling, and molecule representation learning.
APA
Lee, G., Jin, W., Alvarez-Melis, D. & Jaakkola, T.. (2019). Functional Transparency for Structured Data: a Game-Theoretic Approach. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:3723-3733 Available from https://proceedings.mlr.press/v97/lee19b.html.

Related Material