Matching Learned Causal Effects of Neural Networks with Domain Priors

Sai Srinivas Kancheti, Abbavaram Gowtham Reddy, Vineeth N Balasubramanian, Amit Sharma
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:10676-10696, 2022.

Abstract

A trained neural network can be interpreted as a structural causal model (SCM) that provides the effect of changing input variables on the model’s output. However, if training data contains both causal and correlational relationships, a model that optimizes prediction accuracy may not necessarily learn the true causal relationships between input and output variables. On the other hand, expert users often have prior knowledge of the causal relationship between certain input variables and output from domain knowledge. Therefore, we propose a regularization method that aligns the learned causal effects of a neural network with domain priors, including both direct and total causal effects. We show that this approach can generalize to different kinds of domain priors, including monotonicity of causal effect of an input variable on output or zero causal effect of a variable on output for purposes of fairness. Our experiments on twelve benchmark datasets show its utility in regularizing a neural network model to maintain desired causal effects, without compromising on accuracy. Importantly, we also show that a model thus trained is robust and gets improved accuracy on noisy inputs.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-kancheti22a, title = {Matching Learned Causal Effects of Neural Networks with Domain Priors}, author = {Kancheti, Sai Srinivas and Reddy, Abbavaram Gowtham and Balasubramanian, Vineeth N and Sharma, Amit}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {10676--10696}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/kancheti22a/kancheti22a.pdf}, url = {https://proceedings.mlr.press/v162/kancheti22a.html}, abstract = {A trained neural network can be interpreted as a structural causal model (SCM) that provides the effect of changing input variables on the model’s output. However, if training data contains both causal and correlational relationships, a model that optimizes prediction accuracy may not necessarily learn the true causal relationships between input and output variables. On the other hand, expert users often have prior knowledge of the causal relationship between certain input variables and output from domain knowledge. Therefore, we propose a regularization method that aligns the learned causal effects of a neural network with domain priors, including both direct and total causal effects. We show that this approach can generalize to different kinds of domain priors, including monotonicity of causal effect of an input variable on output or zero causal effect of a variable on output for purposes of fairness. Our experiments on twelve benchmark datasets show its utility in regularizing a neural network model to maintain desired causal effects, without compromising on accuracy. Importantly, we also show that a model thus trained is robust and gets improved accuracy on noisy inputs.} }
Endnote
%0 Conference Paper %T Matching Learned Causal Effects of Neural Networks with Domain Priors %A Sai Srinivas Kancheti %A Abbavaram Gowtham Reddy %A Vineeth N Balasubramanian %A Amit Sharma %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-kancheti22a %I PMLR %P 10676--10696 %U https://proceedings.mlr.press/v162/kancheti22a.html %V 162 %X A trained neural network can be interpreted as a structural causal model (SCM) that provides the effect of changing input variables on the model’s output. However, if training data contains both causal and correlational relationships, a model that optimizes prediction accuracy may not necessarily learn the true causal relationships between input and output variables. On the other hand, expert users often have prior knowledge of the causal relationship between certain input variables and output from domain knowledge. Therefore, we propose a regularization method that aligns the learned causal effects of a neural network with domain priors, including both direct and total causal effects. We show that this approach can generalize to different kinds of domain priors, including monotonicity of causal effect of an input variable on output or zero causal effect of a variable on output for purposes of fairness. Our experiments on twelve benchmark datasets show its utility in regularizing a neural network model to maintain desired causal effects, without compromising on accuracy. Importantly, we also show that a model thus trained is robust and gets improved accuracy on noisy inputs.
APA
Kancheti, S.S., Reddy, A.G., Balasubramanian, V.N. & Sharma, A.. (2022). Matching Learned Causal Effects of Neural Networks with Domain Priors. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:10676-10696 Available from https://proceedings.mlr.press/v162/kancheti22a.html.

Related Material