Exploring Conceptual Soundness with TruLens

Anupam Datta, Matt Fredrikson, Klas Leino, Kaiji Lu, Shayak Sen, Ricardo Shih, Zifan Wang
Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, PMLR 176:302-307, 2022.

Abstract

As machine learning has become increasingly ubiquitous, there has been a growing need to assess the trustworthiness of learned models. One important aspect to model trust is conceptual soundness, i.e., the extent to which a model uses features that are appropriate for its intended task. We present TruLens, a new cross-platform framework for explaining deep network behavior. In our demonstration, we provide an interactive application built on TruLens that we use to explore the conceptual soundness of various pre-trained models. We take the unique perspective that robustness to small-norm adversarial examples is a necessary condition for conceptual soundness; we demonstrate this by comparing explanations on models trained with and without a robust objective. Our demonstration will focus on our end-to-end application, which will be made accessible for the audience to interact with; but we will also provide details on its open-source components, including the TruLens library and the code used to train robust networks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v176-datta22a, title = {Exploring Conceptual Soundness with TruLens}, author = {Datta, Anupam and Fredrikson, Matt and Leino, Klas and Lu, Kaiji and Sen, Shayak and Shih, Ricardo and Wang, Zifan}, booktitle = {Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track}, pages = {302--307}, year = {2022}, editor = {Kiela, Douwe and Ciccone, Marco and Caputo, Barbara}, volume = {176}, series = {Proceedings of Machine Learning Research}, month = {06--14 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v176/datta22a/datta22a.pdf}, url = {https://proceedings.mlr.press/v176/datta22a.html}, abstract = {As machine learning has become increasingly ubiquitous, there has been a growing need to assess the trustworthiness of learned models. One important aspect to model trust is conceptual soundness, i.e., the extent to which a model uses features that are appropriate for its intended task. We present TruLens, a new cross-platform framework for explaining deep network behavior. In our demonstration, we provide an interactive application built on TruLens that we use to explore the conceptual soundness of various pre-trained models. We take the unique perspective that robustness to small-norm adversarial examples is a necessary condition for conceptual soundness; we demonstrate this by comparing explanations on models trained with and without a robust objective. Our demonstration will focus on our end-to-end application, which will be made accessible for the audience to interact with; but we will also provide details on its open-source components, including the TruLens library and the code used to train robust networks.} }
Endnote
%0 Conference Paper %T Exploring Conceptual Soundness with TruLens %A Anupam Datta %A Matt Fredrikson %A Klas Leino %A Kaiji Lu %A Shayak Sen %A Ricardo Shih %A Zifan Wang %B Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track %C Proceedings of Machine Learning Research %D 2022 %E Douwe Kiela %E Marco Ciccone %E Barbara Caputo %F pmlr-v176-datta22a %I PMLR %P 302--307 %U https://proceedings.mlr.press/v176/datta22a.html %V 176 %X As machine learning has become increasingly ubiquitous, there has been a growing need to assess the trustworthiness of learned models. One important aspect to model trust is conceptual soundness, i.e., the extent to which a model uses features that are appropriate for its intended task. We present TruLens, a new cross-platform framework for explaining deep network behavior. In our demonstration, we provide an interactive application built on TruLens that we use to explore the conceptual soundness of various pre-trained models. We take the unique perspective that robustness to small-norm adversarial examples is a necessary condition for conceptual soundness; we demonstrate this by comparing explanations on models trained with and without a robust objective. Our demonstration will focus on our end-to-end application, which will be made accessible for the audience to interact with; but we will also provide details on its open-source components, including the TruLens library and the code used to train robust networks.
APA
Datta, A., Fredrikson, M., Leino, K., Lu, K., Sen, S., Shih, R. & Wang, Z.. (2022). Exploring Conceptual Soundness with TruLens. Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, in Proceedings of Machine Learning Research 176:302-307 Available from https://proceedings.mlr.press/v176/datta22a.html.

Related Material