PoRB-Nets: Poisson Process Radial Basis Function Networks

Beau Coker, Melanie Fernandez Pradier, Finale Doshi-Velez
Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), PMLR 124:1338-1347, 2020.

Abstract

Bayesian neural networks (BNNs) are flexible function priors well-suited to situations in which data are scarce and uncertainty must be quantified. Yet, common weight priors are able to encode little functional knowledge and can behave in undesirable ways. We present a novel prior over radial basis function networks (RBFNs) that allows for independent specification of functional amplitude variance and lengthscale (i.e., smoothness), where the inverse lengthscale corresponds to the concentration of radial basis functions. When the lengthscale is uniform over the input space, we prove consistency and approximate variance stationarity. This is in contrast to common BNN priors, which are highly nonstationary. When the input dependence of the lengthscale is unknown, we show how it can be inferred. We compare this model’s behavior to standard BNNs and Gaussian processes using synthetic and real examples.

Cite this Paper

BibTeX
@InProceedings{pmlr-v124-coker20a, title = {PoRB-Nets: Poisson Process Radial Basis Function Networks}, author = {Coker, Beau and Fernandez Pradier, Melanie and Doshi-Velez, Finale}, booktitle = {Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)}, pages = {1338--1347}, year = {2020}, editor = {Peters, Jonas and Sontag, David}, volume = {124}, series = {Proceedings of Machine Learning Research}, month = {03--06 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v124/coker20a/coker20a.pdf}, url = {https://proceedings.mlr.press/v124/coker20a.html}, abstract = {Bayesian neural networks (BNNs) are flexible function priors well-suited to situations in which data are scarce and uncertainty must be quantified. Yet, common weight priors are able to encode little functional knowledge and can behave in undesirable ways. We present a novel prior over radial basis function networks (RBFNs) that allows for independent specification of functional amplitude variance and lengthscale (i.e., smoothness), where the inverse lengthscale corresponds to the concentration of radial basis functions. When the lengthscale is uniform over the input space, we prove consistency and approximate variance stationarity. This is in contrast to common BNN priors, which are highly nonstationary. When the input dependence of the lengthscale is unknown, we show how it can be inferred. We compare this model’s behavior to standard BNNs and Gaussian processes using synthetic and real examples.} }
Endnote
%0 Conference Paper %T PoRB-Nets: Poisson Process Radial Basis Function Networks %A Beau Coker %A Melanie Fernandez Pradier %A Finale Doshi-Velez %B Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI) %C Proceedings of Machine Learning Research %D 2020 %E Jonas Peters %E David Sontag %F pmlr-v124-coker20a %I PMLR %P 1338--1347 %U https://proceedings.mlr.press/v124/coker20a.html %V 124 %X Bayesian neural networks (BNNs) are flexible function priors well-suited to situations in which data are scarce and uncertainty must be quantified. Yet, common weight priors are able to encode little functional knowledge and can behave in undesirable ways. We present a novel prior over radial basis function networks (RBFNs) that allows for independent specification of functional amplitude variance and lengthscale (i.e., smoothness), where the inverse lengthscale corresponds to the concentration of radial basis functions. When the lengthscale is uniform over the input space, we prove consistency and approximate variance stationarity. This is in contrast to common BNN priors, which are highly nonstationary. When the input dependence of the lengthscale is unknown, we show how it can be inferred. We compare this model’s behavior to standard BNNs and Gaussian processes using synthetic and real examples.
APA
Coker, B., Fernandez Pradier, M. & Doshi-Velez, F.. (2020). PoRB-Nets: Poisson Process Radial Basis Function Networks. Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), in Proceedings of Machine Learning Research 124:1338-1347 Available from https://proceedings.mlr.press/v124/coker20a.html.

Related Material