Proceedings of Machine Learning Research

Composing inference algorithms as program transformations

Fri, 11 Aug 2017 00:00:00 +0000

Probabilistic inference procedures are usually coded painstakingly from scratch, for each tar- get model and each inference algorithm. We reduce this effort by generating inference pro- cedures from models automatically. We make this code generation modular by decomposing inference algorithms into reusable program-to- program transformations. These transforma- tions perform exact inference as well as gener- ate probabilistic programs that compute expec- tations, densities, and MCMC samples. The resulting inference procedures are about as ac- curate and fast as other probabilistic program- ming systems on real-world problems.

Holographic Feature Representations of Deep Networks

Fri, 11 Aug 2017 00:00:00 +0000

It is often asserted that deep networks learn “features”, traditionally expressed by the ac- tivations of intermediate nodes. We explore an alternative concept by defining features as partial derivatives of model output with re- spect to model parameters—extending a sim- ple yet powerful idea from generalized linear models. The resulting features are not equiva- lent to node activations, and we show that they can induce a holographic representation of the complete model: the network’s output on given data can be exactly replicated by a simple lin- ear model over such features extracted from any ordered cut. We demonstrate useful advan- tages for this feature representation over stan- dard representations based on node activations.

The total belief theorem

Fri, 11 Aug 2017 00:00:00 +0000

In this paper, motivated by the treatment of conditional constraints in the data association problem, we state and prove the generalisation of the law of total probability to belief func- tions, as finite random sets. Our results apply to the case in which Dempster’s conditioning is employed. We show that the solution to the resulting total belief problem is in general not unique, whereas it is unique when the a-priori belief function is Bayesian. Examples and case studies underpin the theoretical contributions. Finally, our results are compared to previous related work on the generalisation of Jeffrey’s rule by Spies and Smets.

Stochastic L-BFGS Revisited: Improved Convergence Rates and Practical Acceleration Strategies

Fri, 11 Aug 2017 00:00:00 +0000

We revisit the stochastic limited-memory BFGS (L-BFGS) algorithm. By proposing a new frame- work for analyzing convergence, we theoreti- cally improve the (linear) convergence rates and computational complexities of the stochastic L- BFGS algorithms in previous works. In addi- tion, we propose several practical acceleration strategies to speed up the empirical performance of such algorithms. We also provide theoretical analyses for most of the strategies. Experiments on large-scale logistic and ridge regression prob- lems demonstrate that our proposed strategies yield significant improvements via-‘a-vis compet- ing state-of-the-art algorithms.

Improving Optimization-Based Approximate Inference by Clamping Variables

Fri, 11 Aug 2017 00:00:00 +0000

While central to the application of probabilis- tic models to discrete data, the problem of marginal inference is in general intractable and efficient approximation schemes need to exploit the problem structure. Recently, there have been efforts to develop inference techniques that do not necessarily make factorization as- sumptions about the distribution, but rather ex- ploit the fact that sometimes there exist effi- cient algorithms for finding the MAP config- uration. In this paper, we theoretically prove that for discrete multi-label models the bounds on the partition function obtained by two of these approaches, Perturb-and-MAP and the bound from the infinite R{é}nyi divergence, can be only improved by clamping any subset of the variables. For the case of log-supermodular models we provide a more detailed analysis and develop a set of efficient strategies for choos- ing the order in which the variables should be clamped. Finally, we present a number of nu- merical experiments showcasing the improve- ments obtained by the proposed methods on several models.

Feature-to-Feature Regression for a Two-Step Conditional Independence Test

Fri, 11 Aug 2017 00:00:00 +0000

The algorithms for causal discovery and more broadly for learning the structure of graphical models require well calibrated and consistent conditional independence (CI) tests. We revisit the CI tests which are based on two-step pro- cedures and involve regression with subsequent (unconditional) independence test (RESIT) on regression residuals and investigate the assump- tions under which these tests operate. In particu- lar, we demonstrate that when going beyond sim- ple functional relationships with additive noise, such tests can lead to an inflated number of false discoveries. We study the relationship of these tests with those based on dependence measures using reproducing kernel Hilbert spaces (RKHS) and propose an extension of RESIT which uses RKHS-valued regression. The resulting test in- herits the simple two-step testing procedure of RESIT, while giving correct Type I control and competitive power. When used as a component of the PC algorithm, the proposed test is more robust to the case where hidden variables induce a switching behaviour in the associations present in the data.

Determinantal Point Processes for Mini-Batch Diversification

Fri, 11 Aug 2017 00:00:00 +0000

We study a mini-batch diversification scheme for stochastic gradient descent (SGD). While classi- cal SGD relies on uniformly sampling data points to form a mini-batch, we propose a non-uniform sampling scheme based on the Determinantal Point Process (DPP). The DPP relies on a sim- ilarity measure between data points and gives low probabilities to mini-batches which contain redun- dant data, and higher probabilities to mini-batches with more diverse data. This simultaneously bal- ances the data and leads to stochastic gradients with lower variance. We term this approach Di- versified Mini-Batch SGD (DM-SGD). We show that regular SGD and a biased version of stratified sampling emerge as special cases. Furthermore, DM-SGD generalizes stratified sampling to cases where no discrete features exist to bin the data into groups. We show experimentally that our method results more interpretable and diverse features in unsupervised setups, and in better classification accuracies in supervised setups.

SAT-Based Causal Discovery under Weaker Assumptions

Fri, 11 Aug 2017 00:00:00 +0000

Using the flexibility of recently developed methods for causal discovery based on Boolean satisfiability (SAT) solvers, we en- code a variety of assumptions that weaken the Faithfulness assumption. The encoding results in a number of SAT-based algorithms whose asymptotic correctness relies on weaker condi- tions than are standardly assumed. This imple- mentation of a whole set of assumptions in the same platform enables us to systematically ex- plore the effect of weakening the Faithfulness assumption on causal discovery. An important effect, suggested by simulation results, is that adopting weaker assumptions greatly allevi- ates the problem of conflicting constraints and substantially shortens solving time. As a re- sult, SAT-based causal discovery is potentially more scalable under weaker assumptions.

Communication-Efficient Distributed Primal-Dual Algorithm for Saddle Point Problems

Fri, 11 Aug 2017 00:00:00 +0000

Primal-dual algorithms, which are proposed to solve reformulated convex-concave saddle point problems, have been proven to be effec- tive for solving a generic class of convex opti- mization problems, especially when the prob- lems are ill-conditioned. However, the sad- dle point problem still lacks a distributed op- timization framework where primal-dual algo- rithms can be employed. In this paper, we propose a novel communication-efficient dis- tributed optimization framework to solve the convex-concave saddle point problem based on primal-dual methods. We carefully de- sign local subproblems and a central problem such that our proposed distributed optimiza- tion framework is communication-efficient. We provide a convergence analysis of our proposed algorithm, and extend it to ad- dress non-smooth and non-strongly convex loss functions. We conduct extensive experi- ments on several real-world datasets to demon- strate competitive performance of the proposed method, especially on ill-conditioned prob- lems.

Convex-constrained Sparse Additive Modeling and Its Extensions

Fri, 11 Aug 2017 00:00:00 +0000

Sparse additive modeling is a class of effec- tive methods for performing high-dimensional nonparametric regression. In this work we show how shape constraints such as convex- ity/concavity and their extensions, can be in- tegrated into additive models. The proposed sparse difference of convex additive models (SDCAM) can estimate most continuous func- tions without any a priori smoothness assump- tion. Motivated by a characterization of dif- ference of convex functions, our method in- corporates a natural regularization functional to avoid overfitting and to reduce model com- plexity. Computationally, we develop an ef- ficient backfitting algorithm with linear per- iteration complexity. Experiments on both synthetic and real data confirm that our method is competitive against state-of-the-art sparse additive models, with improved performance in most scenarios.

Neighborhood Regularized $\ell^1$-Graph

Fri, 11 Aug 2017 00:00:00 +0000

$\ell$1-Graph, which learns a sparse graph over the data by sparse representation, has been demonstrated to be effective in clustering es- pecially for high dimensional data. Although it achieves compelling performance, the sparse graph generated by $\ell$1-Graph ignores the geo- metric information of the data by sparse rep- resentation for each datum separately. To ob- tain a sparse graph that is aligned to the un- derlying manifold structure of the data, we propose the novel Neighborhood Regularized $\ell$1-Graph (NR$\ell$1-Graph). NR$\ell$1-Graph learns sparse graph with locally consistent neigh- borhood by encouraging nearby data to have similar neighbors in the constructed sparse graph. We present the optimization algorithm of NR$\ell$1-Graph with theoretical guarantee on the convergence and the gap between the sub- optimal solution and the globally optimal so- lution in each step of the coordinate descent, which is essential for the overall optimiza- tion of NR$\ell$1-Graph. Its provable acceler- ated version, NR$\ell$1-Graph by Random Projec- tion (NR$\ell$1-Graph-RP) that employs random- ized data matrix decomposition, is also pre- sented to improve the efficiency of the opti- mization of NR$\ell$1-Graph. Experimental re- sults on various real data sets demonstrate the effectiveness of both NR$\ell$1-Graph and NR$\ell$1- Graph-RP. This work is supported in part by US Army Research Of- fice grant W911NF-15-1-0317. The work of Jiashi Feng was supported by NUS startup R-263-000-C08-133, MOE R-263- 000-C21-112 and IDS R-263-000-C67-646. Pushmeet Kohli was at Microsoft Research during this project.

Decoupling Homophily and Reciprocity with Latent Space Network Models

Fri, 11 Aug 2017 00:00:00 +0000

Networks form useful representations of data arising in various physical and social domains. In this work, we consider dynamic networks such as communication networks in which links connecting pairs of nodes appear over continuous time. We adopt a point process- based approach, and study latent space models which embed the nodes into Euclidean space. We propose models to capture two different as- pects of dynamic network data: (i) commu- nication occurs at a higher rate between indi- viduals with similar features (homophily), and (ii) individuals tend to reciprocate communi- cations from other nodes, but in a manner that varies across individuals. Our framework mar- ries ideas from point process models, includ- ing Poisson and Hawkes processes, with ideas from latent space models of static networks. We evaluate our models over a range of tasks on real-world datasets and show that a dual la- tent space model, which accounts for hetero- geneity in both reciprocity and homophily, sig- nificantly improves performance for both static and dynamic link prediction.

A Fast Stochastic Riemannian Eigensolver

Fri, 11 Aug 2017 00:00:00 +0000

We propose a fast stochastic Riemannian gradient eigensolver for a real and symmetric matrix, and prove its local, eigengap-dependent and linear convergence. The fast convergence is brought by deploying the variance reduction technique which was originally developed for the Euclidean strongly convex problems. In this paper, this technique is generalized to Riemannian manifolds for solving the geodesically non-convex problem of finding a group of top eigenvectors of such a matrix. We first propose the general variance reduction form of the stochastic Riemannian gradient, giving rise to the stochastic variance reduced Riemannian gradient method (SVRRG). It turns out that the operation of vector transport is necessary in addition to using Riemannian gradients and retraction operations. We then specialize it to the problem in question resulting in our SVRRG-EIGS algorithm. We are among the first to propose and analyze the generalization of the stochastic variance reduced gradient (SVRG) to Riemannian manifolds. As an extension of the linearly convergent VR-PCA, it is significant and nontrivial for the proposed algorithm to theoretically achieve a further speedup and empirically make a difference, due to our respect to the inherent geometry of the problem. ID: 49 link sup A Kernel Conditional Independence Test for Relational Data Sanghack Lee; Vasant Honavar Conditional independence (CI) tests play a central role in statistical inference, machine learning, and causal discovery. Most existing CI tests assume that the samples are independently and identically distributed (i.i.d.). However, this assumption often does not hold in the case of relational data. We define Relational Conditional Independence (RCI), a generalization of CI to the relational setting. We show how, under a set of structural assumptions, we can test for RCI by reducing the task of testing for RCI on non-i.i.d. data to the problem of testing for CI on several data sets each of which consists of i.i.d. samples. We develop Kernel Relational CI test (KRCIT), a nonparametric test as a practical approach to testing for RCI by relaxing the structural assumptions used in our analysis of RCI. We describe results of experiments with synthetic relational data that show the benefits of KRCIT relative to traditional CI tests that don’t account for the non-i.i.d. nature of relational data. ID: 171 link sup A Practical Method for Solving Contextual Bandit Problems Using Decision Trees Adam N. Elmachtoub; Ryan McNellis; Sechan Oh; Marek Petrik Many efficient algorithms with strong theoretical guarantees have been proposed for the contextual multi-armed bandit problem. However, applying these algorithms in practice can be difficult because they require domain expertise to build appropriate features and to tune their parameters. We propose a new method for the contextual bandit problem that is simple, practical, and can be applied with little or no domain expertise. Our algorithm relies on decision trees to model the context-reward relationship. Decision trees are non-parametric, interpretable, and work well without hand-crafted features. To guide the exploration-exploitation trade-off, we use a bootstrapping approach which abstracts Thompson sampling to non-Bayesian settings. We also discuss several computational heuristics and demonstrate the performance of our method on several datasets. ID: 278 link A Probabilistic Framework for Zero-Shot Multi-Label Learning Abhilash Gaure; Aishwarya Gupta; Vinay Kumar Verma; Piyush Rai We present a probabilistic framework for multi-label learning for the setting when the test data may require predicting labels that were not available at training time (i.e., the zero-shot learning setting). We develop a probabilistic model that leverages the co-occurrence statistics of the labels via a joint generative model for the label matrix (which denotes the label presence/absence for each training example) and for the label co-occurrence matrix (which denotes how many times a pair of labels co-occurs with each other). In addition to handling the unseen labels at test time, leveraging the co-occurrence information may also help in the standard multi-label learning setting, especially if the number of training examples is very small and/or the label matrix of training examples has a large fraction of missing entries. Our experimental results demonstrate the efficacy of our model in handling unseen labels. ID: 209 link A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units Niranjani Prasad; Li-Fang Cheng; Corey Chivers; Michael Draugelis; Barbara E Engelhardt The management of invasive mechanical ventilation, and the regulation of sedation and analgesia during ventilation, constitutes a major part of the care of patients admitted to intensive care units. Both prolonged dependence on mechanical ventilation and premature extubation are associated with increased risk of complications and higher hospital costs, but clinical opinion on the best protocol for weaning patients off the ventilator varies. This work looks to develop a decision support tool that uses available information to predict time to extubation readiness and recommend a personalized regime of sedation dosage and ventilator support. To this end, we employ off-policy reinforcement learning algorithms to determine the best action at a given patient state from sub-optimal historical ICU data. We compare treatment policies from fitted Q-iteration with extremely randomized trees and with feedforward neural networks, and demonstrate that the policies learnt show promise in recommending weaning protocols with improved outcomes, in terms of minimizing the rate of reintubation and closer regulation of physiological stability. ID: 220 link sup A Tractable Probabilistic Model for Subset Selection Yujia Shen; Arthur Choi; Adnan Darwiche Subset selection tasks, such as top-k ranking, induce datasets where examples have cardinalities that are known a priori. In this paper, we propose a tractable probabilistic model for subset selection and show how it can be learned from data. Our proposed model is interpretable and subsumes a previously introduced model based on logistic regression. We show how the parameters of our model can be estimated in closed form given complete data, and propose an algorithm for learning its structure in an interpretable space. We highlight the intuitive structures that we learn via case studies. We finally show how our proposed model can be viewed as an instance of the recently proposed Probabilistic Sentential Decision Diagram. ID: 306 link sup Adversarial Sets for Regularising Neural Link Predictors Pasquale Minervini; Thomas Demeester; Tim Rockt{ä}schel; Sebastian Riedel In adversarial training, a set of models learn together by pursuing competing goals, usually defined on single data instances. However, in relational learning and other non-i.i.d domains, goals can also be defined over sets of instances. For example, a link predictor for the is-a relation needs to be consistent with the transitivity property: if is-a(x_1, x_2) and is-a(x_2, x_3) hold, is-a(x_1, x_3) needs to hold as well. Here we use such assumptions for deriving an inconsistency loss, measuring the degree to which the model violates the assumptions on an adversarially-generated set of examples. The training objective is defined as a minimax problem, where an adversary finds the most offending adversarial examples by maximising the inconsistency loss, and the model is trained by jointly minimising a supervised loss and the inconsistency loss on the adversarial examples. This yields the first method that can use function-free Horn clauses (as in Datalog) to regularise any neural link predictor, with complexity independent of the domain size. We show that for several link prediction models, the optimisation problem faced by the adversary has efficient closed-form solutions. Experiments on link prediction benchmarks indicate that given suitable prior knowledge, our method can significantly improve neural link predictors on all relevant metrics. ID: 277 link sup Algebraic Equivalence of Linear Structural Equation Models Thijs van Ommen; Joris M. Mooij Despite their popularity, many questions about the algebraic constraints imposed by linear structural equation models remain open problems. For causal discovery, two of these problems are especially important: the enumeration of the constraints imposed by a model, and deciding whether two graphs define the same statistical model. We show how the half-trek criterion can be used to make progress in both of these problems. We apply our theoretical results to a small-scale model selection problem, and find that taking the additional algebraic constraints into account may lead to significant improvements in model selection accuracy. ID: 54 link sup Best Student Paper Honorable Mention

Data-Dependent Sparsity for Subspace Clustering

Fri, 11 Aug 2017 00:00:00 +0000

Subspace clustering is the process of assign- ing subspace memberships to a set of un- labeled data points assumed to have been drawn from the union of an unknown number of low-dimensional subspaces, possibly inter- laced with outliers or other data corruptions. By exploiting the fact that each inlier point has a sparse representation with respect to a dic- tionary formed by all the other points, an $\ell$1 regularized sparse subspace clustering (SSC) method has recently shown state-of-the-art ro- bustness and practical extensibility in a vari- ety of applications. But there remain impor- tant lingering weaknesses. In particular, the $\ell$1 norm solution is highly sensitive, often in a detrimental direction, to the very types of data structures that motivate interest in subspace clustering to begin with, sometimes leading to poor segmentation accuracy. However, as an alternative source of sparsity, we argue that a certain data-dependent, non-convex penalty function can compensate for dictionary struc- ture in a way that is especially germane to sub- space clustering problems. For example, we demonstrate that this proposal displays a form of invariance to feature-space transformations and affine translations that commonly disrupt existing methods, and moreover, in important settings we reveal that its performance quality is lower bounded by the $\ell$1 solution. Finally, we provide empirical comparisons on popu- lar benchmarks that corroborate our theoreti- cal findings and demonstrate superior perfor- mance when compared to recent state-of-the- art models.

Near-Orthogonality Regularization in Kernel Methods

Fri, 11 Aug 2017 00:00:00 +0000

Kernel methods perform nonlinear learning in high-dimensional reproducing kernel Hilbert spaces (RKHSs). Even though their large model-capacity leads to high representational power, it also incurs substantial risk of over- fitting. To alleviate this problem, we pro- pose a new regularization approach, near- orthogonality regularization, which encour- ages the RKHS functions to be close to be- ing orthogonal. This effectively imposes a structural constraint over the function space, which reduces model complexity and can im- prove generalization performance. Besides, encouraging orthogonality reduces the redun- dancy among functions, which hence can re- duce model size without compromising mod- eling power and better capture infrequent pat- terns in the data. Here, we define a family of orthogonality-promoting regularizers by en- couraging the Gram matrix of the RKHS func- tions to be close to an identity matrix where the closeness is measured by Bregman ma- trix divergences. We apply these regularizers to two kernel methods, and develop an effi- cient ADMM-based algorithm to solve the reg- ularized optimization problems. We analyze how near-orthogonality affects the generaliza- tion performance of kernel methods. Our re- sults suggest that the closer the functions are to being orthogonal, the smaller the general- ization error is. Experiments demonstrate the efficacy of near-orthogonality regularization in kernel methods.

Green Generative Modeling: Recycling Dirty Data using Recurrent Variational Autoencoders

Fri, 11 Aug 2017 00:00:00 +0000

This paper explores two useful modifications of the recent variational autoencoder (VAE), a popular deep generative modeling frame- work that dresses traditional autoencoders with probabilistic attire. The first involves a specially-tailored form of conditioning that al- lows us to simplify the VAE decoder struc- ture while simultaneously introducing robust- ness to outliers. In a related vein, a second, complementary alteration is proposed to fur- ther build invariance to contaminated or dirty samples via a data augmentation process that amounts to recycling. In brief, to the extent that the VAE is legitimately a representative generative model, then each output from the decoder should closely resemble an authentic sample, which can then be resubmitted as a novel input ad infinitum. Moreover, this can be accomplished via special recurrent connec- tions without the need for additional parame- ters to be trained. We evaluate these proposals on multiple practical outlier-removal and gen- erative modeling tasks, demonstrating consid- erable improvements over existing algorithms.

Stochastic Bandit Models for Delayed Conversions

Fri, 11 Aug 2017 00:00:00 +0000

Online advertising and product recommenda- tion are important domains of applications for multi-armed bandit methods. In these fields, the reward that is immediately available is most often only a proxy for the actual outcome of interest, which we refer to as a conversion. For instance, in web advertising, clicks can be observed within a few seconds after an ad dis- play but the corresponding sale –if any– will take hours, if not days to happen. This pa- per proposes and investigates a new stochas- tic multi-armed bandit model in the framework proposed by Chapelle (2014) –based on em- pirical studies in the field of web advertising– in which each action may trigger a future re- ward that will then happen with a stochas- tic delay. We assume that the probability of conversion associated with each action is un- known while the distribution of the conversion delay is known, distinguishing between the (idealized) case where the conversion events may be observed whatever their delay and the more realistic setting in which late conversions are censored. We provide performance lower bounds as well as two simple but efficient algo- rithms based on the UCB and KLUCB frame- works. The latter algorithm, which is prefer- able when conversion rates are low, is based on a Poissonization argument, of independent interest in other settings where aggregation of Bernoulli observations with different success probabilities is required.

Efficient solutions for Stochastic Shortest Path Problems with Dead Ends

Fri, 11 Aug 2017 00:00:00 +0000

Many planning problems require maximizing the probability of goal satisfaction as well as min- imizing the expected cost to reach the goal. To model and solve such problems, there have been several attempts at extending Stochastic Shortest Path problems (SSPs) to deal with dead ends and optimize a dual optimization criterion. Unfortu- nately these extensions lack either theoretical ro- bustness or practical efficiency. We study a new, perhaps more natural optimization criterion cap- turing these problems, the Min-Cost given Max- Prob (MCMP) criterion. This criterion leads to the minimum expected cost policy among those with maximum success probability, and accu- rately accounts for the cost and risk of reaching dead ends. Moreover, it lends itself to efficient solution methods that build on recent heuristic search algorithms for the dual representation of stochastic shortest paths problems. Our experi- ments show up to one order of magnitude speed- up over the state of the art.

Safe Semi-Supervised Learning of Sum-Product Networks

Fri, 11 Aug 2017 00:00:00 +0000

In several domains obtaining class annotations is expensive while at the same time unlabelled data are abundant. While most semi-supervised approaches enforce restrictive assumptions on the data distribution, recent work has man- aged to learn semi-supervised models in a non- restrictive regime. However, so far such ap- proaches have only been proposed for linear models. In this work, we introduce semi- supervised parameter learning for Sum-Product Networks (SPNs). SPNs are deep probabilis- tic models admitting inference in linear time in number of network edges. Our approach has several advantages, as it (1) allows genera- tive and discriminative semi-supervised learn- ing, (2) guarantees that adding unlabelled data can increase, but not degrade, the performance (safe), and (3) is computationally efficient and does not enforce restrictive assumptions on the data distribution. We show on a variety of data sets that safe semi-supervised learning with SPNs is competitive compared to state-of-the- art and can lead to a better generative and dis- criminative objective value than a purely super- vised approach.

Branch and Bound for Regular Bayesian Network Structure Learning

Fri, 11 Aug 2017 00:00:00 +0000

We consider efficient Bayesian network structure learning (BNSL) based on scores using branch and bound. Thus far, as a BNSL score, the Bayesian Dirichlet equiva- lent uniform (BDeu) has been used most of- ten, but it is recently proved that the BDeu does not choose the simplest model even when the likelihood is maximized whereas Jeffreys’ prior and MDL satisfy such regu- larity. Although the BDeu has been pre- ferred because it gives Markov equivalent models the same score, in this paper, we introduce another class of scores (quotient scores) that satisfies the property, and pro- pose a pruning rule for the quotient score based on Jeffreys’ prior. We find that the quotient score based on Jeffreys’ prior is regular, and that the proposed pruning rule utilizes the regularity, and is applied much more often than that of the BDeu, so that much less computation is required in BNSL. Finally, our experiments support the hypothesis that the regular scores out- perform the non-regular ones in the sense of computational efficiency as well as cor- rectness of BNSL.

Multi-dueling Bandits with Dependent Arms

Fri, 11 Aug 2017 00:00:00 +0000

The dueling bandits problem is an online learning framework for learning from pairwise preference feedback, and is particularly well- suited for modeling settings that elicit subjec- tive or implicit human feedback. In this paper, we study the problem of multi-dueling bandits with dependent arms, which extends the orig- inal dueling bandits setting by simultaneously dueling multiple arms as well as modeling de- pendencies between arms. These extensions capture key characteristics found in many real- world applications, and allow for the opportu- nity to develop significantly more efficient al- gorithms than were possible in the original set- ting. We propose the SELFSPARRING algo- rithm, which reduces the multi-dueling bandits problem to a conventional bandit setting that can be solved using a stochastic bandit algo- rithm such as Thompson Sampling, and can naturally model dependencies using a Gaus- sian process prior. We present a no-regret anal- ysis for multi-dueling setting, and demonstrate the effectiveness of our algorithm empirically on a wide range of simulation settings.

Treatment-Response Models for Counterfactual Reasoning with Continuous-time, Continuous-valued Interventions

Fri, 11 Aug 2017 00:00:00 +0000

Treatment effects can be estimated from ob- servational data as the difference in poten- tial outcomes. In this paper, we address the challenge of estimating the potential outcome when treatment-dose levels can vary continu- ously over time. Further, the outcome variable may not be measured at a regular frequency. Our proposed solution represents the treatment response curves using linear time-invariant dynamical systems—this provides a flexible means for modeling response over time to highly variable dose curves. Moreover, for multivariate data, the proposed method: un- covers shared structure in treatment response and the baseline across multiple markers; and, flexibly models challenging correlation struc- ture both across and within signals over time. For this, we build upon the framework of multiple-output Gaussian Processes. On sim- ulated and a challenging clinical dataset, we show significant gains in accuracy over state- of-the-art models.

Stochastic Segmentation Trees for Multiple Ground Truths

Fri, 11 Aug 2017 00:00:00 +0000

A strong machine learning system should aim to produce the full range of valid interpreta- tions rather than a single mode in tasks involv- ing inherent ambiguity. This is particularly true for image segmentation, in which there are many sensible ways to partition an image into regions. We formulate a tree-structured probabilistic model, the stochastic segmenta- tion tree, that represents a distribution over segmentations of a given image. We train this model by optimizing a novel objective that quantifies the degree of match between statis- tics of the model and ground truth segmen- tations. Our method allows learning of both the parameters in the tree and the structure itself. We demonstrate on two datasets, in- cluding the challenging Berkeley Segmenta- tion Dataset, that our model is able to success- fully capture the range of ground truths and to produce novel plausible segmentations beyond those found in the data.

Monte-Carlo Tree Search using Batch Value of Perfect Information

Fri, 11 Aug 2017 00:00:00 +0000

This paper focuses on the selection phase of Monte- Carlo Tree Search (MCTS). We define batch value of perfect information (BVPI) in game trees as a gener- alization of value of computation as proposed by Rus- sell and Wefald, and use it for selecting nodes to sam- ple in MCTS. We show that computing the BVPI is NP-hard, but it can be approximated in polynomial time. In addition, we propose methods that intelli- gently find sets of fringe nodes with high BVPI, and quickly select nodes to sample from these sets. We ap- ply our new BVPI methods to partial game trees, both in a stand-alone set of tests, and as a component of a full MCTS algorithm. Empirical results show that our BVPI methods outperform existing node-selection methods for MCTS in different scenarios.

A Tractable Probabilistic Model for Subset Selection

Fri, 11 Aug 2017 00:00:00 +0000

Subset selection tasks, such as top-k ranking, induce datasets where examples have cardinal- ities that are known a priori. In this paper, we propose a tractable probabilistic model for sub- set selection and show how it can be learned from data. Our proposed model is interpretable and subsumes a previously introduced model based on logistic regression. We show how the parameters of our model can be estimated in closed form given complete data, and propose an algorithm for learning its structure in an in- terpretable space. We highlight the intuitive structures that we learn via case studies. We finally show how our proposed model can be viewed as an instance of the recently proposed Probabilistic Sentential Decision Diagram.

Best Student Paper Honorable Mention

An Efficient Minibatch Acceptance Test for Metropolis-Hastings

Fri, 11 Aug 2017 00:00:00 +0000

We present a novel Metropolis-Hastings method for large datasets that uses small expected-size minibatches of data. Previous work on reducing the cost of Metropolis- Hastings tests yield variable data consumed per sample, with only constant factor reductions versus using the full dataset for each sample. Here we present a method that can be tuned to provide arbitrarily small batch sizes, by adjusting either proposal step size or temperature. Our test uses the noise-tolerant Barker acceptance test with a novel additive correction variable. The resulting test has similar cost to a normal SGD update. Our experiments demonstrate several order-of-magnitude speedups over previous work. ID: 180 link sup Analysis of Thompson Sampling for Stochastic Sleeping Bandits Aritra Chatterjee; Ganesh Ghalme; Shweta Jain; Rohit Vaish; Y. Narahari We study a variant of the stochastic multi-armed bandit problem where the set of available arms varies arbitrarily with time (also known as the sleeping bandit problem). We focus on the Thompson Sampling algorithm and consider a regret notion defined with respect to the best available arm. Our main result is an O(log T) regret bound for Thompson Sampling, which generalizes a similar bound known for this algorithm from the classical bandit setting. Our bound also matches (up to constants) the best-known lower bound for the sleeping bandit problem. We show via simulations that Thompson Sampling outperforms the UCB-style AUER algorithm for the sleeping bandit problem. ID: 146 link Approximate Evidential Reasoning Using Local Conditioning and Conditional Belief Functions Van Nguyen We propose a new message-passing belief propagation method that approximates belief updating on evidential networks with conditional belief functions. By means of local conditioning, the method is able to propagate beliefs on the original multiply-connected network structure using local computations, facilitating reasoning in a distributed and dynamic context. Further, by use of conditional belief functions in the form of partially defined plausibility and basic plausibility assignment functions, belief updating can be efficiently approximated using only partial information of the belief functions involved. Experiments show that the method produces results with high degree of accuracy whilst achieving a significant decrease in computational and space complexity (compared to exact methods). ID: 109 link Approximation Complexity of Maximum A Posteriori Inference in Sum-Product Networks Diarmaid Conaty; Denis D. Maua; Cassio P. de Campos We discuss the computational complexity of approximating maximum a posteriori inference in sum-product networks. We first show NP-hardness in trees of height two by a reduction from maximum independent set; this implies non-approximability within a sublinear factor. We show that this is a tight bound, as we can find an approximation within a linear factor in networks of height two. We then show that, in trees of height three, it is NP-hard to approximate the problem within a factor $2^{f(n)}$ for any sublinear function $f$ of the size of the input $n$. Again, this bound is tight, as we prove that the usual max-product algorithm finds (in any network) approximations within factor $2^{c \cdot n}$ for some constant $c ID: 50 link sup AutoGP: Exploring the Capabilities and Limitations of Gaussian Process Models Karl Krauth; Edwin V. Bonilla; Kurt Cutajar; Maurizio Filippone We investigate the capabilities and limitations of Gaussian process (GP) models by jointly exploring three complementary directions: (i) scalable and statistically efficient inference; (ii) flexible kernels; and (iii) objective functions for hyperparameter learning alternative to the marginal likelihood. Our approach outperforms all previous GP methods on the MNIST dataset; performs comparatively to kernel-based methods using the RECTANGLES-IMAGE dataset; and breaks the 1% error-rate barrier in GP models on the MNIST8M dataset, while showing unprecedented scalability (8 million observations) in GP classification. Overall, our approach represents a significant breakthrough in kernel methods and GP models, bridging the gap between deep learning and kernel machines. ID: 70 link sup Bayesian Inference of Log Determinants J Fitzsimons; K Cutajar; M Filippone; M Osborne; S Roberts The log-determinant of a kernel matrix appears in a variety of machine learning problems, ranging from determinantal point processes and generalized Markov random fields, through to the training of Gaussian processes. Exact calculation of this term is often intractable when the size of the kernel matrix exceeds a few thousands. In the spirit of probabilistic numerics, we reinterpret the problem of computing the log-determinant as a Bayesian inference problem. In particular, we combine prior knowledge in the form of bounds from matrix theory and evidence derived from stochastic trace estimation to obtain probabilistic estimates for the log-determinant and its associated uncertainty within a given computational budget. Beyond its novelty and theoretic appeal, the performance of our proposal is competitive with state-of-the-art approaches to approximating the log-determinant, while also quantifying the uncertainty due to budget-constrained evidence. ID: 78 link sup Branch and Bound for Regular Bayesian Network Structure Learning Joe Suzuki; Jun Kawahara We consider efficient Bayesian network structure learning (BNSL) using branch and bound. Thus far, it is known that efficient pruning rules have been found for minimizing the description length (MDL) rather than for maximizing the posterior probability such as the Bayesian Dirichlet equivalent uniform (BDeu) that has been used most often. In any BNSL, given data, the simplest structure should be chosen if structures are equally fit to the data. We show that quotient scores satisfy such regularity while the BDeu does not. In this paper, by utilizing the regularity, we propose an pruning rule for maximizing the posterior probability based on the quotient scores, and prove that the bound is tighter than existing one. We examine by experiments that the proposed bound is applied significantly more often than existing one for the BDeu as well. Besides, we prove that the MDL satisfies regularity and that the proposed bound behaves like that of the MDL, and observe that the proposed bound is almost as efficient as one for the MDL. Our novel insight is that BNSL can be efficient by utilizing regularity for constructing a pruning rule. ID: 11 link sup Causal Consistency of Structural Equation Models Paul K. Rubenstein*; Sebastian Weichwald*; Stephan Bongers; Joris M. Mooij; Dominik Janzing; Moritz Grosse-Wentrup; Bernhard Schoelkopf Complex systems can be modelled at various levels of detail. Ideally, causal models of the same system should be consistent with one another in the sense that they agree in their predictions of the effects of interventions. We formalise this notion of consistency in the case of Structural Equation Models (SEMs) by introducing exact transformations between SEMs. This provides a general language to consider, for instance, the different levels of description in the following three scenarios: (a) models with large numbers of variables versus models in which the ‘irrelevant’ or unobservable variables have been marginalised out; (b) micro-level models versus macro-level models in which the macro-variables are aggregate features of the micro-variables; (c) dynamical time series models versus models of their stationary behaviour. Our analysis stresses the importance of well specified interventions in the causal modelling process and sheds light on the interpretation of cyclic SEMs. ID: 269 link Causal Discovery from Temporally Aggregated Time Series Mingming Gong; Kun Zhang; Bernhard Sch{ö}lkopf; Clark Glymour; Dacheng Tao Discovering causal structure of a dynamical system from observed time series is a traditional and important problem. In many practical applications, observed data are obtained by applying subsampling or temporally aggregation to the original causal processes, making it difficult to discover the underlying causal relations. Subsampling refers to the procedure that for every $k$ consecutive observations, one is kept, the rest being skipped, and recently some advances have been made in causal discovery from such data. With temporal aggregation, the local averages or sums of $k$ consecutive, non-overlapping observations in the causal process are computed as new observations, and causal discovery from such data is even harder. In this paper, we investigate how to recover causal relations at the original causal frequency from temporally aggregated data. Assuming the time series at the causal frequency follows a vector autoregressive (VAR) model, we show that the causal structure at the causal frequency is identifiable from aggregated time series if the noise terms are independent and non-Gaussian and some other technical conditions hold. We then present an estimation method based on non-Gaussian state-space modeling and evaluate its performance on both synthetic and real data. ID: 286 link sup Communication-Efficient Distributed Primal-Dual Algorithm for Saddle Point Problems Yaodong Yu; Sulin Liu; Sinno Jialin Pan Primal-dual algorithms, which are proposed to solve reformulated convex-concave saddle point problems, have been proven to be effective for solving a generic class of convex optimization problems, especially when the problems are ill-conditioned. However, the saddle point problem still lacks a distributed optimization framework where primal-dual algorithms can be employed. In this paper, we propose a novel communication-efficient distributed optimization framework to solve the convex-concave saddle point problem based on primal-dual methods. We carefully design local subproblems and a central problem such that our proposed distributed optimization framework is communication-efficient. We provide a convergence analysis of our proposed algorithm, and extend it to address non-smooth and non-strongly convex loss functions. We conduct extensive experiments on several real-world datasets to demonstrate competitive performance of the proposed method, especially on ill-conditioned problems. ID: 64 link Complexity of Solving Decision Trees with Skew-Symmetric Bilinear Utility Hugo Gilbert; Olivier Spanjaard We study the complexity of solving decision trees with a Skew-Symmetric Bilinear (SSB) utility function. The SSB model is an extension of Expected Utility (EU) with enhanced descriptive possibilities. Unlike EU, the optimality principle does not hold for SSB, which makes its optimization trickier. We show that determining an SSB optimal plan is NP-hard if one only considers deterministic plans while it is polynomial time if one allows randomized plans. With the Weighted EU model (a special case of SSB), the problem becomes polynomial in both settings. Our numerical tests show the operationality of the methods. ID: 163 link Composing inference algorithms as program transformations Robert Zinkov; Chung-chieh Shan Probabilistic inference procedures are usually coded painstakingly from scratch, for each target model and each inference algorithm. We reduce this effort by generating inference procedures from models automatically. We make this code generation modular by decomposing inference algorithms into reusable program-to-program transformations. These transformations perform exact inference as well as generate probabilistic programs that compute expectations, densities, and MCMC samples. The resulting inference procedures are about as accurate and fast as other probabilistic programming systems on real-world problems. ID: 173 link sup Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data Gintare Karolina Dziugaite; Daniel M. Roy One of the defining properties of deep learning is that models are chosen to have many more parameters than available training data. In light of this capacity for overfitting, it is remarkable that simple algorithms like SGD reliably return solutions with low test error. One roadblock to explaining these phenomena in terms of implicit regularization, structural properties of the solution, and/or easiness of the data is that many learning bounds are quantitatively vacuous when applied to networks learned by SGD in this "deep learning" regime. Logically, in order to explain generalization, we need nonvacuous bounds. We return to an idea by Langford and Caruana (2001), who used PAC-Bayes bounds to compute nonvacuous numerical bounds on generalization error for stochastic two-layer two-hidden-unit neural networks via a sensitivity analysis. By optimizing the PAC-Bayes bound directly, we are able to extend their approach and obtain nonvacuous generalization bounds for deep stochastic neural network classifiers with millions of parameters trained on only tens of thousands of examples. We connect our findings to recent and old work on flat minima and MDL-based explanations of generalization. ID: 289 link sup Continuously tempered Hamiltonian Monte Carlo Matthew M Graham; Amos J Storkey Hamiltonian Monte Carlo (HMC) is a powerful Markov chain Monte Carlo (MCMC) method for performing approximate inference in complex probabilistic models of continuous variables. In common with many MCMC methods, however, the standard HMC approach performs poorly in distributions with multiple isolated modes. We present a method for augmenting the Hamiltonian system with an extra continuous temperature control variable which allows the dynamic to bridge between sampling a complex target distribution and a simpler unimodal base distribution. This augmentation both helps improve mixing in multimodal targets and allows the normalisation constant of the target distribution to be estimated. The method is simple to implement within existing HMC code, requiring only a standard leapfrog integrator. We demonstrate experimentally that the method is competitive with annealed importance sampling and simulating tempering methods at sampling from challenging multimodal distributions and estimating their normalising constants. ID: 185 link Convex-constrained Sparse Additive Modeling and Its Extensions Junming Yin; Yaoliang Yu Sparse additive modeling is a class of effective methods for performing high-dimensional nonparametric regression. In this work we show how shape constraints such as convexity/concavity and their extensions, can be integrated into additive models. The proposed sparse difference of convex additive models (SDCAM) can estimate most continuous functions without any a priori smoothness assumption. Motivated by a characterization of difference of convex functions, our method incorporates a natural regularization functional to avoid overfitting and to reduce model complexity. Computationally, we develop an efficient backfitting algorithm with linear per-iteration complexity. Experiments on both synthetic and real data verify that our method is competitive against state-of-the-art sparse additive models, with improved performance in most scenarios. ID: 97 link Counting Markov Equivalence Classes by Number of Immoralities Adityanarayanan Radhakrishnan; Liam Solus; Caroline Uhler Two directed acyclic graphs (DAGs) are called Markov equivalent if and only if they have the same underlying undirected graph (i.e. skeleton) and the same set of immoralities. When using observational data alone and typical identifiability assumptions, such as faithfulness, a DAG model can only be determined up to Markov equivalence. Therefore, it is desirable to understand the size and number of Markov equivalence classes (MECs) combinatorially. In this paper, we address this enumerative question using a pair of generating functions that encode the number and size of MECs on a skeleton G, and in doing so we connect this problem to classical problems in combinatorial optimization. The first generating function is a graph polynomial that counts the number of MECs on G by their number of immoralities. Using connections to the independent set problem, we show that computing a DAG on G with the maximum possible number of immoralities is NP-hard. The second generating function counts the MECs on G according to their size. Via computer enumeration, we show that this generating function is distinct for every connected graph on p nodes for all $p\leq 10$. ID: 141 link sup Coupling Adaptive Batch Sizes with Learning Rates Lukas Balles; Javier Romero; Philipp Hennig Mini-batch stochastic gradient descent and variants thereof have become standard for large-scale empirical risk minimization like the training of neural networks. These methods are usually used with a constant batch size chosen by simple empirical inspection. The batch size significantly influences the behavior of the stochastic optimization algorithm, though, since it determines the variance of the gradient estimates. This variance also changes over the optimization process; when using a constant batch size, stability and convergence is thus often enforced by means of a (manually tuned) decreasing learning rate schedule. We propose a practical method for dynamic batch size adaptation. It estimates the variance of the stochastic gradients and adapts the batch size to decrease the variance proportionally to the value of the objective function, removing the need for the aforementioned learning rate decrease. In contrast to recent related work, our algorithm couples the batch size to the learning rate, directly reflecting the known relationship between the two. On popular image classification benchmarks, our batch size adaptation yields faster optimization convergence, while simultaneously simplifying learning rate tuning. A TensorFlow implementation is available. ID: 93 link sup Data-Dependent Sparsity for Subspace Clustering Bo Xin; Yizhou Wang; Wen Gao and David Wipf Subspace clustering is the process of assigning subspace memberships to a set of unlabeled data points assumed to have been drawn from the union of an unknown number of low-dimensional subspaces, possibly interlaced with outliers or other data corruptions. By exploiting the fact that each inlier point has a sparse representation with respect to a dictionary formed by all the other points, an $\ell_1$ regularized sparse subspace clustering (SSC) method has recently shown state-of-the-art robustness and practical extensibility in a variety of applications. But there remain important lingering weaknesses. In particular, the $\ell_1$ norm solution is highly sensitive, often in a detrimental direction, to the very types of data structures that motivate interest in subspace clustering to begin with, sometimes leading to poor segmentation accuracy. However, as an alternative source of sparsity, we argue that a certain data-dependent, non-convex penalty function can compensate for dictionary structure in a way that is especially germane to subspace clustering problems. For example, we demonstrate that this proposal displays a form of invariance to feature-space transformations and affine translations that commonly disrupt existing methods, and moreover, in important settings we reveal that its performance quality is lower bounded by the $\ell_1$ solution. Finally, we provide empirical comparisons on popular benchmarks that corroborate our theoretical findings and demonstrate superior performance when compared to recent state-of-the-art models. ID: 147 link sup Decoupling Homophily and Reciprocity with Latent Space Network Models Jiasen Yang; Vinayak Rao; Jennifer Neville Networks form useful representations of data arising in various physical and social domains. In this work, we consider dynamic networks such as communication networks in which links connecting pairs of nodes appear over continuous time. We adopt a point process-based approach, and study latent space models which embed the nodes into Euclidean space. We propose models to capture two different aspects of dynamic network data: (i) communication occurs at a higher rate between individuals with similar features (homophily), and (ii) individuals tend to reciprocate communications from other nodes, but in a manner that varies across individuals. Our framework marries ideas from point process models, including Poisson and Hawkes processes, with ideas from latent space models of static networks. We evaluate our models over a range of tasks on real-world datasets and show that a dual latent space model, which accounts for heterogeneity in both reciprocity and homophily, significantly improves performance for both static and dynamic link prediction. ID: 297 link Deep Hybrid Models: Bridging Discriminative and Generative Approaches Volodymyr Kuleshov; Stefano Ermon Most methods in machine learning are described as either discriminative or generative. The former often attain higher predictive accuracy, while the latter are more strongly regularized and can deal with missing data. Here, we propose a new framework to combine a broad class of discriminative and generative models, interpolating between the two extremes with a multiconditional likelihood objective. Unlike previous approaches, we couple the two components through shared latent variables, and train using recent advances in variational inference. Instantiating our framework with modern deep architectures gives rise to deep hybrid models, a highly flexible family that generalizes several existing models and is effective in the semi-supervised setting, where it results in improvements over the state of the art on the SVHN dataset. ID: 69 link sup Determinantal Point Processes for Mini-Batch Diversification Cheng Zhang; Hedvig Kjellstrom; Stephan Mandt We study a mini-batch diversification scheme for stochastic gradient descent (SGD). While classical SGD relies on uniformly sampling data points to form a mini-batch, we propose a non-uniform sampling scheme based on the Determinantal Point Process (DPP). The DPP relies on a similarity measure between data points and gives low probabilities to mini-batches which contain redundant data, and higher probabilities to mini-batches with more diverse data. This simultaneously balances the data and leads to stochastic gradients with lower variance. We term this approach Diversified Mini-Batch SGD (DM-SGD). We show that regular SGD and a biased version of stratified sampling emerge as special cases. Furthermore, DM-SGD generalizes stratified sampling to cases where no discrete features exist to bin the data into groups. We show experimentally that our method results more interpretable and diverse features in unsupervised setups, and in better classification accuracies in supervised setups. ID: 152 link Differentially Private Variational Inference for Non-conjugate Models Joonas J{ä}lk{ö}; Onur Dikmen; Antti Honkela Many machine learning applications are based on data collected from people, such as their tastes and behaviour as well as biological traits and genetic data. Regardless of how important the application might be, one has to make sure individuals’ identities or the privacy of the data are not compromised in the analysis. Differential privacy constitutes a powerful framework that prevents breaching of data subject privacy from the output of a computation. Differentially private versions of many important Bayesian inference methods have been proposed, but there is a lack of an efficient unified approach applicable to arbitrary models. In this contribution, we propose a differentially private variational inference method with a very wide applicability. It is built on top of doubly stochastic variational inference, a recent advance which provides a variational solution to a large class of models. We add differential privacy into doubly stochastic variational inference by clipping and perturbing the gradients. The algorithm is made more efficient through privacy amplification from subsampling. We demonstrate the method can reach an accuracy close to non-private level under reasonably strong privacy guarantees, clearly improving over previous sampling-based alternatives especially in the strong privacy regime. ID: 81 link sup Effective sketching methods for value function approximation Yangchen Pan; Erfan Sadeqi Azer; Martha White High-dimensional representations, such as radial basis function networks or tile coding, are common choices for policy evaluation in reinforcement learning. Learning with such high-dimensional representations, however, can be expensive, particularly for matrix methods, such as least-squares temporal difference learning or quasi-Newton methods that approximate matrix step-sizes. In this work, we explore the utility of sketching for these two classes of algorithms. We highlight issues with sketching the high-dimensional features directly, which can incur significant bias. As a remedy, we demonstrate how to use sketching more sparingly, with only a left-sided sketch, that can still enable significant computational gains and the use of these matrix-based learning algorithms that are less sensitive to parameters. We empirically investigate these algorithms, in four domains with a variety of representations. Our aim is to provide insights into effective use of sketching in practice. ID: 83 link sup Efficient Online Learning for Optimizing Value of Information: Theory and Application to Interactive Troubleshooting Yuxin Chen; Jean-Michel Renders; Morteza Haghir Chehreghani; Andreas Krause We consider the optimal value of information (VoI) problem, where the goal is to sequentially select a set of tests with a minimal cost, so that one can efficiently make the best decision based on the observed outcomes. Existing algorithms are either heuristics with no guarantees, or scale poorly (with exponential run time in terms of the number of available tests). Moreover, these methods assume a known distribution over the test outcomes, which is often not the case in practice. We propose an efficient sampling-based online learning framework to address the above issues. First, assuming the distribution over hypotheses is known, we propose a dynamic hypothesis enumeration strategy, which allows efficient information gathering with strong theoretical guarantees. We show that with sufficient amount of samples, one can identify a near-optimal decision with high probability. Second, when the parameters of the hypotheses distribution are unknown, we propose an algorithm which learns the parameters progressively via posterior sampling in an online fashion. We further establish a rigorous bound on the expected regret. We demonstrate the effectiveness of our approach on a real-world interactive troubleshooting application and show that one can efficiently make high-quality decisions with low cost. ID: 280 link Efficient solutions for Stochastic Shortest Path Problems with Dead Ends Felipe Trevizan; Florent Teichteil-K{ö}nigsbuch; Sylvie Thiebaux Many planning problems require maximizing the probability of goal satisfaction as well as minimizing the expected cost to reach the goal. To model and solve such problems, there has been several attempts at extending stochastic shortest path problems (SSPs) to deal with dead-ends and optimize a dual optimization criterion. Unfortunately these extensions lack either theoretical robustness or practical efficiency. We study a new, perhaps even more natural optimization criterion capturing these problems, the Min-Cost given Max-Prob (MCMP) criterion. This criterion leads to the minimum expected cost policy among those with maximum success probability, and accurately accounts for the cost and risk of reaching dead-ends. Moreover, it lends itself to efficient solution methods that build on recent heuristic search algorithms for the dual representation of stochastic shortest paths problems. Our experiments show up to one order of magnitude speed-up over the state of the art. ID: 135 link Embedding Senses via Dictionary Bootstrapping Byungkon Kang; Kyung-Ah Sohn This paper addresses the problem of embedding senses of a plain word according to its context. Natural language is inherently ambiguous, due to the presence of many multi-sensed words. Such ambiguity might have undesirable influence over many text-mining tools, including word embedding. Traditional word embedding techniques have focused on identifying which words tend to co-occur with one another in order to derive close embeddings for such words. However, the effectiveness of this approach is largely susceptible to the validity and neutrality of the training corpus. To address this problem, we propose to use the dictionary as the authoritative corpus for computing the word embeddings. The basic idea is to simultaneously embed the definition sentence while disambiguating words. Since dictionaries list a set of disambiguated senses, the proposed procedure yields sense embeddings which exhibit semantic characteristics comparable to plain word embeddings. ID: 242 link sup Exact Inference for Relational Graphical Models with Interpreted Functions: Lifted Probabilistic Inference Modulo Theories Rodrigo de Salvo Braz; Ciaran O’Reilly Probabilistic Inference Modulo Theories (PIMT) is a recent framework that expands exact inference on graphical models to use richer languages that include arithmetic, equalities, and inequalities on both integers and real numbers. In this paper, we expand PIMT to a lifted version that also processes random functions and relations. This enhancement is achieved by adapting Inversion, a method from Lifted First-Order Probabilistic Inference literature, to also be modulo theories. This results in the first algorithm for exact probabilistic inference that efficiently and simultaneously exploits random relations and functions, arithmetic, equalities and inequalities. ID: 12 link sup FROSH: FasteR Online Sketching Hashing Xixian Chen; Irwin King; Michael R. Lyu Many hashing methods, especially those that are in the data-dependent category with good learning accuracy, are still inefficient when dealing with three critical problems in modern data analysis. First, data usually come in a streaming fashion, but most of the existing hashing methods are batch-based models. Second, when data become huge, the extensive computational time, large space requirement, and multiple passes to load the data into memory will be prohibitive. Third, data often lack sufficient label information. Although the recently proposed Online Sketching Hashing (OSH) is promising to alleviate all three issues mentioned above, its training procedure still suffers from a high time complexity. In this paper, we propose a FasteR Online Sketching Hashing (FROSH) method to make the training process faster. Compared with OSH, we leverage fast transform to sketch data more compactly. Particularly, we derive independent transformations to guarantee the sketching accuracy, and design a novel implementation to make such transformations applicable to online data sketching without increasing the space cost. We rigorously prove that our method can yield a comparable learning accuracy with a lower time complexity and an equal space cost compared with OSH. Finally, extensive experiments on synthetic and real-world datasets demonstrate the excellent performance of our method. ID: 207 link sup Fair Optimal Stopping Policy for Matching with Mediator Yang Liu In this paper we study an optimal stopping policy for a multi-agent delegated sequential matching system with fairness constraints. We consider a setting where a mediator/decision maker matches a sequence of arriving assignments to multiple groups of agents, with agents being grouped according to certain sensitive attributes that needs to be protected. The decision maker aims to maximize total rewards that can be collected from above matching process (from all groups), while making the matching fair among groups. We discuss two types of fairness constraints: (i) each group has a certain expected deadline before which the match needs to happen; (ii) each group would like to have a guaranteed share of average reward from the matching. We present the exact characterization of fair optimal strategies. Example is provided to demonstrate the computation efficiency of our solution. ID: 75 link sup Fast Amortized Inference and Learning in Log-linear Models with Randomly Perturbed Nearest Neighbor Search Stephen Mussmann; Daniel Levy; Stefano Ermon Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks when the output space is feasibly enumerable but very large. We propose a method to perform inference in log-linear models with sublinear amortized cost. Our idea hinges on using Gumbel random variable perturbations and a pre-computed Maximum Inner Product Search data structure to access the most-likely elements in sublinear amortized time. Our method yields provable runtime and accuracy guarantees. Further, we present empirical experiments on ImageNet and Word Embeddings showing significant speedups for sampling, inference, and learning in log-linear models. ID: 250 link sup Feature-to-Feature Regression for a Two-Step Conditional Independence Test Qinyi Zhang; Sarah Filippi; Seth Flaxman; Dino Sejdinovic The algorithms for causal discovery and more broadly for learning the structure of graphical models require well calibrated and consistent conditional independence (CI) tests. We revisit the CI tests which are based on two-step procedures and involve regression with subsequent (unconditional) independence test (RESIT) on regression residuals and investigate the assumptions under which these tests operate. In particular, we demonstrate that when going beyond simple functional relationships with additive noise, such tests can lead to an inflated number of false discoveries. We study the relationship of these tests with those based on dependence measures using reproducing kernel Hilbert spaces (RKHS) and propose an extension of RESIT which uses RKHS-valued regression. The resulting test inherits the simple two-step testing procedure of RESIT, while giving correct Type I control and competitive power. When used as a component of the PC algorithm, the proposed test is more robust to the case where hidden variables induce a switching behaviour in the associations present in the data. ID: 142 link sup Green Generative Modeling: Recycling Dirty Data using Recurrent Variational Autoencoders Yu Wang; Bin Dai; Gang Hua; John Aston; David Wipf This paper explores two useful modifications of the recent variational autoencoder (VAE), a popular deep generative modeling framework that dresses traditional autoencoders with probabilistic attire. The first involves a specially-tailored form of conditioning that allows us to simplify the VAE decoder structure while simultaneously introducing robustness to outliers. In a related vein, a second, complementary alteration is proposed to further build invariance to contaminated or dirty samples via a data augmentation process that amounts to recycling. In brief, to the extent that the VAE is legitimately a representative generative model, then each output from the decoder should closely resemble an authentic sample, which can then be resubmitted as a novel input ad infinitum. Moreover, this can be accomplished via special recurrent connections without the need for additional parameters to be trained. We evaluate these proposals on multiple practical outlier-removal and generative modeling tasks, demonstrating considerable improvements over existing algorithms. ID: 267 link sup Holographic Feature Representations of Deep Networks Martin A. Zinkevich; Alex Davies; Dale Schuurmans It is often asserted that deep networks learn “"features", traditionally expressed by the activations of intermediate nodes. We explore an alternative concept by defining features as partial derivatives of model output with respect to model parameters—extending a simple yet powerful idea from generalized linear models. The resulting features are not equivalent to node activations, and we show that they can induce a holographic representation of the complete model: the network’s output on given data can be exactly replicated by a simple linear model over such features extracted from any ordered cut. We demonstrate useful advantages for this feature representation over standard representations based on node activations. ID: 255 link sup How Good Are My Predictions? Efficiently Approximating Precision-Recall Curves for Massive Datasets Ashish Sabharwal; Hanie Sedghi Large scale machine learning produces massive datasets whose items are often associated with a confidence level and can thus be ranked. However, computing the precision of these resources requires human annotation, which is often prohibitively expensive and is therefore skipped. We consider the problem of cost-effectively approximating precision-recall (PR) or ROC curves for such systems. Our novel approach, called PAULA, provides theoretically guaranteed lower and upper bounds on the underlying precision function while relying on only O(log N) annotations for a resource with N items. This contrasts favorably with Theta(sqrt(N \log N)) annotations needed by commonly used sampling based methods. Our key insight is to capitalize on a natural monotonicity property of the underlying confidence-based ranking. PAULA provides tight bounds for PR curves using, e.g., only 17K annotations for resources with 200K items and 48K annotations for resources with 2B items. We use PAULA to evaluate a subset of the much utilized PPDB paraphrase database and a recent Science knowledge base. ID: 138 link Importance Sampled Stochastic Optimization for Variational Inference Joseph Sakaya; Arto Klami Variational inference approximates the posterior distribution of a probabilistic model with a parameterized density by maximizing a lower bound for the model evidence. Modern solutions fit a flexible approximation with stochastic gradient descent, using Monte Carlo approximation for the gradients. This enables variational inference for arbitrary differentiable probabilistic models, and consequently makes variational inference feasible for probabilistic programming languages. In this work we develop more efficient inference algorithms for the task by considering importance sampling estimates for the gradients. We show how the gradient with respect to the approximation parameters can often be evaluated efficiently without needing to re-compute gradients of the model itself, and then proceed to derive practical algorithms that use importance sampled estimates to speed up computation. We present importance sampled stochastic gradient descent that outperforms standard stochastic gradient descent by a clear margin for a range of models, and provide a justifiable variant of stochastic average gradients for variational inference. ID: 225 link sup Best Paper

Real-Time Resource Allocation for Tracking Systems

Fri, 11 Aug 2017 00:00:00 +0000

Automated tracking is key to many computer vision applications. However, many tracking systems struggle to perform in real-time due to the high computational cost of detecting peo- ple, especially in ultra high resolution images. We propose a new algorithm called PartiMax that greatly reduces this cost by applying the person detector only to the relevant parts of the image. PartiMax exploits information in the particle filter to select k of the n candidate pixel boxes in the image. We prove that Parti- Max is guaranteed to make a near-optimal se- lection with error bounds that are independent of the problem size. Furthermore, empirical re- sults on a real-life dataset show that our system runs in real-time by processing only 10% of the pixel boxes in the image while still retain- ing 80% of the original tracking performance achieved when processing all pixel boxes.

Importance Sampled Stochastic Optimization for Variational Inference

Fri, 11 Aug 2017 00:00:00 +0000

Variational inference approximates the poste- rior distribution of a probabilistic model with a parameterized density by maximizing a lower bound for the model evidence. Modern solu- tions fit a flexible approximation with stochastic gradient descent, using Monte Carlo approxima- tion for the gradients. This enables variational inference for arbitrary differentiable probabilis- tic models, and consequently makes variational inference feasible for probabilistic programming languages. In this work we develop more effi- cient inference algorithms for the task by consid- ering importance sampling estimates for the gra- dients. We show how the gradient with respect to the approximation parameters can often be eval- uated efficiently without needing to re-compute gradients of the model itself, and then proceed to derive practical algorithms that use impor- tance sampled estimates to speed up computa- tion. We present importance sampled stochas- tic gradient descent that outperforms standard stochastic gradient descent by a clear margin for a range of models, and provide a justifiable vari- ant of stochastic average gradients for variational inference.

How Good Are My Predictions? Efficiently Approximating Precision-Recall Curves for Massive Datasets

Fri, 11 Aug 2017 00:00:00 +0000

Large scale machine learning produces mas- sive datasets whose items are often associ- ated with a confidence level and can thus be ranked. However, computing the precision of these resources requires human annotation, which is often prohibitively expensive and is therefore skipped. We consider the problem of cost-effectively approximating precision- recall (PR) or ROC curves for such sys- tems. Our novel approach, called PAULA, pro- vides theoretically guaranteed lower and up- per bounds on the underlying precision func- tion while relying on only O(log N) anno- tations for a resource with N items. This contrasts favorably with $\Theta$($\sqrt{}$N log N) anno- tations needed by commonly used sampling based methods. Our key insight is to capital- ize on a natural monotonicity property of the underlying confidence-based ranking. PAULA provides tight bounds for PR curves using, e.g., only 17K annotations for resources with 200K items and 48K annotations for resources with 2B items. We use PAULA to evaluate a subset of the much utilized PPDB paraphrase database and a recent Science knowledge base.

Causal Consistency of Structural Equation Models

Fri, 11 Aug 2017 00:00:00 +0000

Complex systems can be modelled at various levels of detail. Ideally, causal models of the same system should be consistent with one an- other in the sense that they agree in their pre- dictions of the effects of interventions. We for- malise this notion of consistency in the case of Structural Equation Models (SEMs) by intro- ducing exact transformations between SEMs. This provides a general language to consider, for instance, the different levels of description in the following three scenarios: (a) models with large numbers of variables versus models in which the ‘irrelevant’ or unobservable variables have been marginalised out; (b) micro-level models versus macro-level models in which the macro- variables are aggregate features of the micro- variables; (c) dynamical time series models ver- sus models of their stationary behaviour. Our analysis stresses the importance of well speci- fied interventions in the causal modelling pro- cess and sheds light on the interpretation of cyclic SEMs.

Counting Markov Equivalence Classes by Number of Immoralities

Fri, 11 Aug 2017 00:00:00 +0000

Two directed acyclic graphs (DAGs) are called Markov equivalent if and only if they have the same underlying undirected graph (i.e. skele- ton) and the same set of immoralities. When using observational data alone and typical identifiability assumptions, such as faithful- ness, a DAG model can only be determined up to Markov equivalence. Therefore, it is de- sirable to understand the size and number of Markov equivalence classes (MECs) combina- torially. In this paper, we address this enu- merative question using a pair of generating functions that encode the number and size of MECs on a skeleton G, and in doing so we connect this problem to classical problems in combinatorial optimization. The first generat- ing function is a graph polynomial that counts the number of MECs on G by their number of immoralities. Using connections to the inde- pendent set problem, we show that computing a DAG on G with the maximum possible num- ber of immoralities is NP-hard. The second generating function counts the MECs on G ac- cording to their size. Via computer enumera- tion, we show that this generating function is distinct for every connected graph on p nodes for all p $\leq$10.

Learning to Acquire Information

Fri, 11 Aug 2017 00:00:00 +0000

We consider the problem of diagnosis where a set of simple observations are used to in- fer a potentially complex hidden hypothesis. Finding the optimal subset of observations is intractable in general, thus we focus on the problem of active diagnosis, where the agent selects the next most-informative observation based on the results of previous observations. We show that under the assumption of uniform observation entropy, one can build an impli- cation model which directly predicts the out- come of the potential next observation condi- tioned on the results of past observations, and selects the observation with the maximum en- tropy. This approach enjoys reduced computa- tion complexity by bypassing the complicated hypothesis space, and can be trained on obser- vation data alone, learning how to query with- out knowledge of the hidden hypothesis.

A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units

Fri, 11 Aug 2017 00:00:00 +0000

The management of invasive mechanical venti- lation, and the regulation of sedation and anal- gesia during ventilation, constitutes a major part of the care of patients admitted to intensive care units. Both prolonged dependence on mechan- ical ventilation and premature extubation are as- sociated with increased risk of complications and higher hospital costs, but clinical opinion on the best protocol for weaning patients off of a ven- tilator varies. This work aims to develop a de- cision support tool that uses available patient in- formation to predict time-to-extubation readiness and to recommend a personalized regime of seda- tion dosage and ventilator support. To this end, we use off-policy reinforcement learning algo- rithms to determine the best action at a given pa- tient state from sub-optimal historical ICU data. We compare treatment policies from fitted Q- iteration with extremely randomized trees and with feedforward neural networks, and demon- strate that the policies learnt show promise in recommending weaning protocols with improved outcomes, in terms of minimizing rates of reintu- bation and regulating physiological stability.

Interpreting and using CPDAGs with background knowledge

Fri, 11 Aug 2017 00:00:00 +0000

We develop terminology and methods for working with maximally oriented partially directed acyclic graphs (maximal PDAGs). Maximal PDAGs arise from imposing restric- tions on a Markov equivalence class of directed acyclic graphs, or equivalently on its graph- ical representation as a completed partially directed acyclic graph (CPDAG), for exam- ple when adding background knowledge about certain edge orientations. Although maximal PDAGs often arise in practice, causal meth- ods have been mostly developed for CPDAGs. In this paper, we extend such methodology to maximal PDAGs. In particular, we de- velop methodology to read off possible ances- tral relationships, we introduce a graphical cri- terion for covariate adjustment to estimate total causal effects, and we adapt the IDA and joint- IDA frameworks to estimate multi-sets of pos- sible causal effects. We also present a simula- tion study that illustrates the gain in identifia- bility of total causal effects as the background knowledge increases. All methods are imple- mented in the R package pcalg.

Near-Optimal Interdiction of Factored MDPs

Fri, 11 Aug 2017 00:00:00 +0000

Stackelberg games have been widely used to model interactions between attackers and de- fenders in a broad array of security domains. One related approach involves plan interdic- tion, whereby a defender chooses a subset of actions to block (remove), and the attacker constructs an optimal plan in response. In pre- vious work, this approach has been introduced in the context of Markov decision processes (MDPs). The key challenge, however, is that the state space of MDPs grows exponentially in the number of state variables. We propose a novel scalable MDP interdiction framework which makes use of factored representation of state, using a parity function basis for repre- senting a value function over a Boolean space. We demonstrate that our approach is signifi- cantly more scalable than prior art, while re- sulting in near-optimal interdiction decisions.

Effective sketching methods for value function approximation

Fri, 11 Aug 2017 00:00:00 +0000

High-dimensional representations, such as ra- dial basis function networks or tile cod- ing, are common choices for policy evalua- tion in reinforcement learning. Learning with such high-dimensional representations, how- ever, can be expensive, particularly for matrix methods, such as least-squares temporal differ- ence learning or quasi-Newton methods that approximate matrix step-sizes. In this work, we explore the utility of sketching for these two classes of algorithms. We highlight issues with sketching the high-dimensional features directly, which can incur significant bias. As a remedy, we demonstrate how to use sketching more sparingly, with only a left-sided sketch, that can still enable significant computational gains and the use of these matrix-based learn- ing algorithms that are less sensitive to param- eters. We empirically investigate these algo- rithms, in four domains with a variety of repre- sentations. Our aim is to provide insights into effective use of sketching in practice.

Iterative Decomposition Guided Variable Neighborhood Search for Graphical Model Energy Minimization

Fri, 11 Aug 2017 00:00:00 +0000

Graphical models factorize a global probabil- ity distribution/energy function as the prod- uct/sum of local functions. A major infer- ence task, known as MAP in Markov Ran- dom Fields and MPE in Bayesian Networks, is to find a global assignment of all the vari- ables with maximum a posteriori probabil- ity/minimum energy. A usual distinction on MAP solving methods is complete/incomplete, i.e. the ability to prove optimality or not. Most complete methods rely on tree search, while incomplete methods rely on local search. Among them, we study Variable Neighbor- hood Search (VNS) for graphical models. In this paper, we propose an iterative approach above VNS which uses (partial) tree search in- side its local neighborhood exploration. The resulting hybrid method offers a good compro- mise between completeness and anytime be- havior than existing tree search methods while still being competitive for proving optimality. We further propose a parallel version of our method improving its anytime behavior on dif- ficult instances coming from a large graphical model benchmark. Last we experiment on the challenging minimum energy problem found in Computational Protein Design, showing the practical benefit of our parallel version. Solver at www.inra.fr/mia/T/toulbar2 v1.0.

Algebraic Equivalence of Linear Structural Equation Models

Fri, 11 Aug 2017 00:00:00 +0000

Despite their popularity, many questions about the algebraic constraints imposed by linear structural equation models remain open prob- lems. For causal discovery, two of these prob- lems are especially important: the enumeration of the constraints imposed by a model, and de- ciding whether two graphs define the same sta- tistical model. We show how the half-trek cri- terion can be used to make progress in both of these problems. We apply our theoretical results to a small-scale model selection prob- lem, and find that taking the additional alge- braic constraints into account may lead to sig- nificant improvements in model selection ac- curacy.

Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels

Fri, 11 Aug 2017 00:00:00 +0000

$\tilde{P}$ $\tilde{N}$ learning is the problem of binary classi- fication when training examples may be mis- labeled (flipped) uniformly with noise rate $\rho$1 for positive examples and $\rho$0 for negative ex- amples. We propose Rank Pruning (RP) to solve $\tilde{P}$ $\tilde{N}$ learning and the open problem of es- timating the noise rates. Unlike prior solutions, RP is efficient and general, requiring O(T) for any unrestricted choice of probabilistic classi- fier with T fitting time. We prove RP achieves consistent noise estimation and equivalent ex- pected risk as learning with uncorrupted labels in ideal conditions, and derive closed-form so- lutions when conditions are non-ideal. RP achieves state-of-the-art noise estimation and F1, error, and AUC-PR for both MNIST and CIFAR datasets, regardless of the amount of noise. To highlight, RP with a CNN classifier can predict if an MNIST digit is a one or not with only 0.25% error, and 0.46% error across all digits, even when 50% of positive examples are mislabeled and 50% of observed positive labels are mislabeled negative examples.

Provable Inductive Robust PCA via Iterative Hard Thresholding

Fri, 11 Aug 2017 00:00:00 +0000

Online Constrained Model-based Reinforcement Learning

Fri, 11 Aug 2017 00:00:00 +0000

Applying reinforcement learning to robotic systems poses a number of challenging prob- lems. A key requirement is the ability to han- dle continuous state and action spaces while remaining within a limited time and resource budget. Additionally, for safe operation, the system must make robust decisions under hard constraints. To address these challenges, we propose a model based approach that com- bines Gaussian Process regression and Reced- ing Horizon Control. Using sparse spectrum Gaussian Processes, we extend previous work by updating the dynamics model incrementally from a stream of sensory data. This results in an agent that can learn and plan in real-time under non-linear constraints. We test our ap- proach on a cart pole swing-up environment and demonstrate the benefits of online learn- ing on an autonomous racing task. The envi- ronment’s dynamics are learned from limited training data and can be reused in new task in- stances without retraining.

Supervised Restricted Boltzmann Machines

Fri, 11 Aug 2017 00:00:00 +0000

We propose in this paper the supervised re- stricted Boltzmann machine (sRBM), a unified framework which combines the versatility of RBM to simultaneously learn the data represen- tation and to perform supervised learning (i.e., a nonlinear classifier or a nonlinear regressor). Un- like the current state-of-the-art classification for- mulation proposed for RBM in (Larochelle et al., 2012), our model is a hybrid probabilistic graph- ical model consisting of a distinguished genera- tive component for data representation and a dis- criminative component for prediction. While the work of (Larochelle et al., 2012) typically incurs no extra difficulty in inference compared with a standard RBM, our discriminative component, modeled as a directed graphical model, renders MCMC-based inference (e.g., Gibbs sampler) very slow and unpractical for use. To this end, we further develop scalable variational inference for the proposed sRBM for both classification and regression cases. Extensive experiments on real- world datasets show that our sRBM achieves bet- ter predictive performance than baseline meth- ods. At the same time, our proposed framework yields learned representations which are more discriminative, hence interpretable, than those of its counterparts. Besides, our method is prob- abilistic and capable of generating meaningful data conditioning on specific classes – a topic which is of current great interest in deep learn- ing aiming at data generation.

Approximate Evidential Reasoning Using Local Conditioning and Conditional Belief Functions

Fri, 11 Aug 2017 00:00:00 +0000

We propose a new message-passing belief propa- gation method that approximates belief updating on evidential networks with conditional belief functions. By means of local conditioning, the method is able to propagate beliefs on the origi- nal multiply-connected network structure using local computations, facilitating reasoning in a distributed and dynamic context. Further, by use of conditional belief functions in the form of par- tially defined plausibility and basic plausibility assignment functions, belief updating can be ef- ficiently approximated using only partial infor- mation of the belief functions involved. Exper- iments show that the method produces results with high degree of accuracy whilst achieving a significant decrease in computational and space complexity (compared to exact methods).

Learning Approximately Objective Priors

Fri, 11 Aug 2017 00:00:00 +0000

Informative Bayesian priors are often difficult to elicit, and when this is the case, modelers usually turn to noninformative or objective pri- ors. However, objective priors such as the Jef- freys and reference priors are not tractable to derive for many models of interest. We address this issue by proposing techniques for learn- ing reference prior approximations: we select a parametric family and optimize a black-box lower bound on the reference prior objective to find the member of the family that serves as a good approximation. We experimentally demonstrate the method’s effectiveness by re- covering Jeffreys priors and learning the Vari- ational Autoencoder’s reference prior.

Fast Amortized Inference and Learning in Log-linear Models with Randomly Perturbed Nearest Neighbor Search

Fri, 11 Aug 2017 00:00:00 +0000

Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks when the output space is feasibly enumerable but very large. We propose a method to per- form inference in log-linear models with sub- linear amortized cost. Our idea hinges on using Gumbel random variable perturbations and a pre-computed Maximum Inner Product Search data structure to access the most-likely elements in sublinear amortized time. Our method yields provable runtime and accuracy guarantees. Further, we present empirical ex- periments on ImageNet and Word Embeddings showing significant speedups for sampling, in- ference, and learning in log-linear models.

Adversarial Sets for Regularising Neural Link Predictors

Fri, 11 Aug 2017 00:00:00 +0000

In adversarial training, a set of models learn together by pursuing competing goals, usu- ally defined on single data instances. How- ever, in relational learning and other non-i.i.d domains, goals can also be defined over sets of instances. For example, a link predictor for the IS-A relation needs to be consistent with the transitivity property: if IS-A(x1, x2) and IS-A(x2, x3) hold, IS-A(x1, x3) needs to hold as well. Here we use such assumptions for deriving an inconsistency loss, measuring the degree to which the model violates the assumptions on an adversarially-generated set of examples. The training objective is de- fined as a minimax problem, where an adver- sary finds the most offending adversarial ex- amples by maximising the inconsistency loss, and the model is trained by jointly minimis- ing a supervised loss and the inconsistency loss on the adversarial examples. This yields the first method that can use function-free Horn clauses (as in Datalog) to regularise any neu- ral link predictor, with complexity independent of the domain size. We show that for several link prediction models, the optimisation prob- lem faced by the adversary has efficient closed- form solutions. Experiments on link predic- tion benchmarks indicate that given suitable prior knowledge, our method can significantly improve neural link predictors on all relevant metrics.

Synthesis of strategies in influence diagrams

Fri, 11 Aug 2017 00:00:00 +0000

Influence diagrams (IDs) are a powerful tool for representing and solving decision problems un- der uncertainty. The objective of evaluating an ID is to compute the expected utility and an opti- mal strategy, which consists of a policy for each decision. Every policy is usually represented as a table containing a column for each decision sce- nario, i.e., for each configuration of the variables on which it depends. The no-forgetting assump- tion, which implies that the decision maker al- ways remembers all past observations and de- cisions, makes the policies grow exponentially with the number of variables in the ID. For hu- man experts it is very difficult to understand the strategy contained in huge policy tables, not only for their size, but also because the vast major- ity of columns correspond to suboptimal or im- possible scenarios and are hence irrelevant. This makes it difficult to extract the rules of action, to debug the model, and to convince the experts that the recommendations of the ID are reasonable. In this paper, we propose a method that presents the strategy in the form of a compact tree. It has been implemented in OpenMarkov, an open- source software tool for probabilistic graphical models. This facility was essential when eval- uating an influence diagram for the mediastinal staging of non-small cell lung cancer; the op- timal strategy, whose biggest policy table con- tained more than 15,000 columns, was synthe- sized into a tree of only 5 leaves.

Stein Variational Policy Gradient

Fri, 11 Aug 2017 00:00:00 +0000

Policy gradient methods have been successfully applied to many complex reinforcement learn- ing problems. However, policy gradient meth- ods suffer from high variance, slow convergence, and inefficient exploration. In this work, we in- troduce a maximum entropy policy optimization framework which explicitly encourages param- eter exploration, and show that this framework can be reduced to a Bayesian inference problem. We then propose a novel Stein variational pol- icy gradient method (SVPG) which combines ex- isting policy gradient methods and a repulsive functional to generate a set of diverse but well- behaved policies. SVPG is robust to random ini- tializations and can easily be implemented in a parallel manner. On several continuous control problems, we find that SVPG versions of RE- INFORCE and advantage actor-critic algorithms are greatly improved in terms of both average re- turn and data efficiency.

Fair Optimal Stopping Policy for Matching with Mediator

Fri, 11 Aug 2017 00:00:00 +0000

In this paper we study an optimal stopping policy for a multi-agent delegated sequential matching system with fairness constraints. We consider a setting where a mediator/decision maker matches a sequence of arriving assign- ments to multiple groups of agents, with agents being grouped according to certain sensitive attributes that needs to be protected. The deci- sion maker aims to maximize total rewards that can be collected from above matching process (from all groups), while making the matching fair among groups. We discuss two types of fairness constraints: (i) each group has a cer- tain expected deadline before which the match needs to happen; (ii) each group would like to have a guaranteed share of average reward from the matching. We present the exact char- acterization of fair optimal strategies. Example is provided to demonstrate the computation ef- ficiency of our solution.

Shortest Path under Uncertainty: Exploration versus Exploitation

Fri, 11 Aug 2017 00:00:00 +0000

In the Canadian Traveler Problem (CTP), a traveler seeks a shortest path to a destination through a road network, but unknown to the traveler, some roads may be blocked. This paper studies the Bayesian CTP (BCTP), in which road states are correlated with known prior probabilities and the traveler can in- fer the states of an unseen road from past observations of other correlated roads. As generalized shortest-path problems, CTP and BCTP have important practical applications. We show that BCTP is NP-complete and give a polynomial-time approximation algorithm, Hedged Shortest Path under Determinization (HSPD), which approximates an optimal solu- tion with a polylogarithmic factor. Preliminary experiments show promising results. HSPD outperforms a widely used greedy algorithm and a state-of-the-art UCT-based search algo- rithm for CTP, especially when significant ex- ploration is required.

Learning the Structure of Probabilistic Sentential Decision Diagrams

Fri, 11 Aug 2017 00:00:00 +0000

The probabilistic sentential decision diagram (PSDD) was recently introduced as a tractable representation of probability distributions that are subject to logical constraints. Meanwhile, efforts in tractable learning achieved great suc- cess inducing complex joint distributions from data without constraints, while guaranteeing efficient exact probabilistic inference; for in- stance by learning arithmetic circuits (ACs) or sum-product networks (SPNs). This pa- per studies the efficacy of PSDDs for the stan- dard tractable learning task without constraints and develops the first PSDD structure learn- ing algorithm, called LEARNPSDD. Experi- ments on standard benchmarks show competi- tive performance, despite the fact that PSDDs are more tractable and more restrictive than their alternatives. LEARNPSDD compares fa- vorably to SPNs, particularly in terms of model size, which is a proxy for tractability. We re- port state-of-the-art likelihood results on six datasets. Moreover, LEARNPSDD retains the ability to learn PSDD structures in probability spaces subject to logical constraints, which is beyond the reach of other representations.

Triply Stochastic Gradients on Multiple Kernel Learning

Fri, 11 Aug 2017 00:00:00 +0000

Multiple Kernel Learning (MKL) is highly useful for learning complex data with multi- ple cues or representations. However, MKL is known to have poor scalability because of the expensive kernel computation. Dai et al (2014) proposed to use a doubly Stochastic Gradient Descent algorithm (doubly SGD) to greatly improve the scalability of kernel meth- ods. However, the algorithm is not suitable for MKL because it cannot learn the kernel weights. In this paper, we provide a novel ex- tension to doubly SGD for MKL so that both the decision functions and the kernel weights can be learned simultaneously. To achieve this, we develop the triply Stochastic Gradient De- scent (triply SGD) algorithm which involves three sources of randomness – the data points, the random features, and the kernels, which was not considered in previous work. We prove that our algorithm enjoys similar conver- gence rate as that of doubly SGD. Comparing to several traditional MKL solutions, we show that our method has faster convergence speed and achieved better accuracy. Most impor- tantly, our method makes it possible to learn MKL problems with millions of data points on a normal desktop PC.

Self-Discrepancy Conditional Independence Test

Fri, 11 Aug 2017 00:00:00 +0000

Tests of conditional independence (CI) of ran- dom variables play an important role in ma- chine learning and causal inference. Of partic- ular interest are kernel-based CI tests which allow us to test for independence among ran- dom variables with complex distribution func- tions. The efficacy of a CI test is measured in terms of its power and its calibratedness. We show that the Kernel CI Permutation Test (KCIPT) suffers from a loss of calibratedness as its power is increased by increasing the number of bootstraps. To address this limita- tion, we propose a novel CI test, called Self- Discrepancy Conditional Independence Test (SDCIT). SDCIT uses a test statistic that is a modified unbiased estimate of maximum mean discrepancy (MMD), the largest difference in the means of features of the given sample and its permuted counterpart in the kernel-induced Hilbert space. We present results of experi- ments that demonstrate SDCIT is, relative to the other methods: (i) competitive in terms of its power and calibratedness, outperforming other methods when the number of condition- ing variables is large; (ii) more robust with re- spect to the choice of the kernel function; and (iii) competitive in run time.

A Kernel Conditional Independence Test for Relational Data

Fri, 11 Aug 2017 00:00:00 +0000

Conditional independence (CI) tests play a central role in statistical inference, machine learning, and causal discovery. Most existing CI tests assume that the samples are indepen- dently and identically distributed (i.i.d.). How- ever, this assumption often does not hold in the case of relational data. We define Relational Conditional Independence (RCI), a generaliza- tion of CI to the relational setting. We show how, under a set of structural assumptions, we can test for RCI by reducing the task of test- ing for RCI on non-i.i.d. data to the problem of testing for CI on several data sets each of which consists of i.i.d. samples. We develop Kernel Relational CI test (KRCIT), a nonpara- metric test as a practical approach to testing for RCI by relaxing the structural assumptions used in our analysis of RCI. We describe re- sults of experiments with synthetic relational data that show the benefits of KRCIT relative to traditional CI tests that don’t account for the non-i.i.d. nature of relational data.

Deep Hybrid Models: Bridging Discriminative and Generative Approaches

Fri, 11 Aug 2017 00:00:00 +0000

Most methods in machine learning are described as either discriminative or generative. The for- mer often attain higher predictive accuracy, while the latter are more strongly regularized and can deal with missing data. Here, we propose a new framework to combine a broad class of discriminative and generative models, interpo- lating between the two extremes with a multi- conditional likelihood objective. Unlike previ- ous approaches, we couple the two components through shared latent variables, and train using recent advances in variational inference. Instanti- ating our framework with modern deep architec- tures gives rise to deep hybrid models, a highly flexible family that generalizes several existing models and is effective in the semi-supervised setting, where it results in improvements over the state of the art on the SVHN dataset.

AutoGP: Exploring the Capabilities and Limitations of Gaussian Process Models

Fri, 11 Aug 2017 00:00:00 +0000

We investigate the capabilities and limita- tions of Gaussian process (GP) models by jointly exploring three complementary direc- tions: (i) scalable and statistically efficient inference; (ii) flexible kernels; and (iii) ob- jective functions for hyperparameter learning alternative to the marginal likelihood. Our approach outperforms all previous GP meth- ods on the MNIST dataset; performs com- paratively to kernel-based methods using the RECTANGLES-IMAGE dataset; and breaks the 1% error-rate barrier in GP models on the MNIST8M dataset, while showing unprece- dented scalability (8 million observations) in GP classification. Overall, our approach rep- resents a significant breakthrough in kernel methods and GP models, bridging the gap be- tween deep learning and kernel machines.

On Loopy Belief Propagation – Local Stability Analysis for Non-Vanishing Fields

Fri, 11 Aug 2017 00:00:00 +0000

In this work we obtain all fixed points of be- lief propagation and perform a local stability analysis. We consider pairwise interactions of binary random variables and investigate the in- fluence of non-vanishing fields and finite-size graphs on the performance of belief propaga- tion; local stability is heavily influenced by these properties. We show why non-vanishing fields help to achieve convergence and increase the accuracy of belief propagation. We fur- ther explain the close connections between the underlying graph structure, the existence of multiple solutions, and the capability of belief propagation (with damping) to converge. Fi- nally we provide insights into why finite-size graphs behave better than infinite-size graphs.

Embedding Senses via Dictionary Bootstrapping

Fri, 11 Aug 2017 00:00:00 +0000

This paper addresses the problem of embedding senses of a plain word according to its context. Natural language is inherently ambiguous, due to the presence of many multi-sensed words. Such ambiguity might have undesirable influence over many text-mining tools, including word embed- ding. Traditional word embedding techniques have focused on identifying which words tend to co-occur with one another in order to derive close embeddings for such words. However, the effectiveness of this approach is largely suscep- tible to the validity and neutrality of the train- ing corpus. To address this problem, we propose to use the dictionary as the authoritative corpus for computing the word embeddings. The ba- sic idea is to simultaneously embed the defini- tion sentence while disambiguating words. Since dictionaries list a set of disambiguated senses, the proposed procedure yields sense embeddings which exhibit semantic characteristics compara- ble to plain word embeddings.

Inverse Reinforcement Learning via Deep Gaussian Process

Fri, 11 Aug 2017 00:00:00 +0000

We propose a new approach to inverse rein- forcement learning (IRL) based on the deep Gaussian process (deep GP) model, which is capable of learning complicated reward struc- tures with few demonstrations. Our model stacks multiple latent GP layers to learn ab- stract representations of the state feature space, which is linked to the demonstrations through the Maximum Entropy learning framework. In- corporating the IRL engine into the nonlinear latent structure renders existing deep GP infer- ence approaches intractable. To tackle this, we develop a non-standard variational approxima- tion framework which extends previous infer- ence schemes. This allows for approximate Bayesian treatment of the feature space and guards against overfitting. Carrying out rep- resentation and inverse reinforcement learn- ing simultaneously within our model outper- forms state-of-the-art approaches, as we demon- strate with experiments on standard bench- marks (“object world”,“highway driving”) and a new benchmark (“binary world”).

Differentially Private Variational Inference for Non-conjugate Models

Fri, 11 Aug 2017 00:00:00 +0000

Many machine learning applications are based on data collected from people, such as their tastes and behaviour as well as biological traits and genetic data. Regardless of how impor- tant the application might be, one has to make sure individuals’ identities or the privacy of the data are not compromised in the analy- sis. Differential privacy constitutes a power- ful framework that prevents breaching of data subject privacy from the output of a com- putation. Differentially private versions of many important Bayesian inference methods have been proposed, but there is a lack of an efficient unified approach applicable to arbi- trary models. In this contribution, we pro- pose a differentially private variational infer- ence method with a very wide applicability. It is built on top of doubly stochastic varia- tional inference, a recent advance which pro- vides a variational solution to a large class of models. We add differential privacy into dou- bly stochastic variational inference by clipping and perturbing the gradients. The algorithm is made more efficient through privacy amplifi- cation from subsampling. We demonstrate the method can reach an accuracy close to non- private level under reasonably strong privacy guarantees, clearly improving over previous sampling-based alternatives especially in the strong privacy regime. $*$AH is also with the Department of Mathematics and Statistics and Department of Public Health, University of Helsinki.

Probabilistic Program Abstractions

Fri, 11 Aug 2017 00:00:00 +0000

Abstraction is a fundamental tool for reasoning about complex systems. Program abstraction has been utilized to great effect for analyzing deterministic programs. At the heart of pro- gram abstraction is the relationship between a concrete program, which is difficult to ana- lyze, and an abstract program, which is more tractable. Program abstractions, however, are typically not probabilistic. We general- ize non-deterministic program abstractions to probabilistic program abstractions by explic- itly quantifying the non-deterministic choices. Our framework upgrades key definitions and properties of abstractions to the probabilistic context. We also discuss preliminary ideas for performing inference on probabilistic abstrac- tions and general probabilistic programs.

Stein Variational Adaptive Importance Sampling

Fri, 11 Aug 2017 00:00:00 +0000

We propose a novel adaptive importance sam- pling algorithm which incorporates Stein vari- ational gradient decent algorithm (SVGD) with importance sampling (IS). Our algorithm lever- ages the nonparametric transforms in SVGD to iteratively decrease the KL divergence between importance proposals and target distributions. The advantages of our algorithm are twofold: 1) it turns SVGD into a standard IS algorithm, al- lowing us to use standard diagnostic and ana- lytic tools of IS to evaluate and interpret the re- sults, and 2) it does not restrict the choice of the importance proposals to predefined distribu- tion families like traditional (adaptive) IS meth- ods. Empirical experiments demonstrate that our algorithm performs well on evaluating partition functions of restricted Boltzmann machines and testing likelihood of variational auto-encoders.

Continuously tempered Hamiltonian Monte Carlo

Fri, 11 Aug 2017 00:00:00 +0000

Hamiltonian Monte Carlo (HMC) is a powerful Markov chain Monte Carlo (MCMC) method for performing approximate inference in com- plex probabilistic models of continuous vari- ables. In common with many MCMC methods, however, the standard HMC approach performs poorly in distributions with multiple isolated modes. We present a method for augmenting the Hamiltonian system with an extra continu- ous temperature control variable which allows the dynamic to bridge between sampling a com- plex target distribution and a simpler unimodal base distribution. This augmentation both helps improve mixing in multimodal targets and al- lows the normalisation constant of the target distribution to be estimated. The method is sim- ple to implement within existing HMC code, re- quiring only a standard leapfrog integrator. We demonstrate experimentally that the method is competitive with annealed importance sampling and simulating tempering methods at sampling from challenging multimodal distributions and estimating their normalising constants.

Causal Discovery from Temporally Aggregated Time Series

Fri, 11 Aug 2017 00:00:00 +0000

Discovering causal structure of a dynamical sys- tem from observed time series is a traditional and important problem. In many practical ap- plications, observed data are obtained by apply- ing subsampling or temporally aggregation to the original causal processes, making it difficult to discover the underlying causal relations. Subsam- pling refers to the procedure that for every k con- secutive observations, one is kept, the rest being skipped, and recently some advances have been made in causal discovery from such data. With temporal aggregation, the local averages or sums of k consecutive, non-overlapping observations in the causal process are computed as new obser- vations, and causal discovery from such data is even harder. In this paper, we investigate how to recover causal relations at the original causal fre- quency from temporally aggregated data when k is known. Assuming the time series at the causal frequency follows a vector autoregressive (VAR) model, we show that the causal structure at the causal frequency is identifiable from aggregated time series if the noise terms are independent and non-Gaussian and some other technical conditions hold. We then present an estimation method based on non-Gaussian state-space modeling and eval- uate its performance on both synthetic and real data.

Complexity of Solving Decision Trees with Skew-Symmetric Bilinear Utility

Fri, 11 Aug 2017 00:00:00 +0000

We study the complexity of solving decision trees with a Skew-Symmetric Bilinear (SSB) utility function. The SSB model is an exten- sion of Expected Utility (EU) with enhanced descriptive possibilities. Unlike EU, the opti- mality principle does not hold for SSB, which makes its optimization trickier. We show that determining an SSB optimal plan is NP-hard if one only considers deterministic plans while it is polynomial time if one allows randomized plans. With the Weighted EU model (a spe- cial case of SSB), the problem becomes poly- nomial in both settings. Our numerical tests show the operationality of the methods.

A Probabilistic Framework for Zero-Shot Multi-Label Learning

Fri, 11 Aug 2017 00:00:00 +0000

We present a probabilistic framework for multi-label learning for the setting when the test data may require predicting labels that were not available at training time (i.e., the zero-shot learning setting). We de- velop a probabilistic model that leverages the co-occurrence statistics of the labels via a joint generative model for the label matrix (which denotes the label presence/absence for each training example) and for the label co- occurrence matrix (which denotes how many times a pair of labels co-occurs with each other). In addition to handling the unseen la- bels at test time, leveraging the co-occurrence information may also help in the standard multi-label learning setting, especially if the number of training examples is very small and/or the label matrix of training examples has a large fraction of missing entries. Our ex- perimental results demonstrate the efficacy of our model in handling unseen labels.

Bayesian Inference of Log Determinants

Fri, 11 Aug 2017 00:00:00 +0000

The log determinant of a kernel matrix ap- pears in a variety of machine learning prob- lems, ranging from determinantal point pro- cesses and generalized Markov random fields, through to the training of Gaussian processes. Exact calculation of this term is often in- tractable when the size of the kernel matrix ex- ceeds a few thousands. In the spirit of proba- bilistic numerics, we reinterpret the problem of computing the log determinant as a Bayesian inference problem. In particular, we com- bine prior knowledge in the form of bounds from matrix theory and evidence derived from stochastic trace estimation to obtain proba- bilistic estimates for the log determinant and its associated uncertainty within a given com- putational budget. Beyond its novelty and the- oretic appeal, the performance of our proposal is competitive with state-of-the-art approaches to approximating the log determinant, while also quantifying the uncertainty due to budget- constrained evidence.

Learning to Draw Samples with Amortized Stein Variational Gradient Descent

Fri, 11 Aug 2017 00:00:00 +0000

We propose a simple algorithm to train stochas- tic neural networks to draw samples from given target distributions for probabilistic inference. Our method is based on iteratively adjusting the neural network parameters so that the output changes along a Stein variational gradient di- rection (Liu & Wang, 2016) that maximally de- creases the KL divergence with the target distri- bution. Our method works for any target dis- tribution specified by their unnormalized density function, and can train any black-box architec- tures that are differentiable in terms of the pa- rameters we want to adapt. We demonstrate our method with a number of applications, including variational autoencoder (VAE) with expressive encoders to model complex latent space struc- tures, and hyper-parameter learning of MCMC samplers that allows Bayesian inference to adap- tively improve itself when seeing more data.

A Practical Method for Solving Contextual Bandit Problems Using Decision Trees

Fri, 11 Aug 2017 00:00:00 +0000

Many efficient algorithms with strong theoreti- cal guarantees have been proposed for the con- textual multi-armed bandit problem. However, applying these algorithms in practice can be difficult because they require domain exper- tise to build appropriate features and to tune their parameters. We propose a new method for the contextual bandit problem that is sim- ple, practical, and can be applied with little or no domain expertise. Our algorithm relies on decision trees to model the context-reward re- lationship. Decision trees are non-parametric, interpretable, and work well without hand- crafted features. To guide the exploration- exploitation trade-off, we use a bootstrapping approach which abstracts Thompson sampling to non-Bayesian settings. We also discuss several computational heuristics and demon- strate the performance of our method on sev- eral datasets.

Structure Learning of Linear Gaussian Structural Equation Models with Weak Edges

Fri, 11 Aug 2017 00:00:00 +0000

We consider structure learning of linear Gaus- sian structural equation models with weak edges. Since the presence of weak edges can lead to a loss of edge orientations in the true underlying CPDAG, we define a new graph- ical object that can contain more edge orien- tations. We show that this object can be re- covered from observational data under a type of strong faithfulness assumption. We present a new algorithm for this purpose, called ag- gregated greedy equivalence search (AGES), that aggregates the solution path of the greedy equivalence search (GES) algorithm for vary- ing values of the penalty parameter. We prove consistency of AGES and demonstrate its per- formance in a simulation study and on single cell data from Sachs et al. (2005). The algo- rithm will be made available in the R-package pcalg.

Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data

Fri, 11 Aug 2017 00:00:00 +0000

One of the defining properties of deep learn- ing is that models are chosen to have many more parameters than available training data. In light of this capacity for overfitting, it is remarkable that simple algorithms like SGD re- liably return solutions with low test error. One roadblock to explaining these phenomena in terms of implicit regularization, structural prop- erties of the solution, and/or easiness of the data is that many learning bounds are quan- titatively vacuous when applied to networks learned by SGD in this “deep learning” regime. Logically, in order to explain generalization, we need nonvacuous bounds. We return to an idea by Langford and Caruana (2001), who used PAC-Bayes bounds to compute nonvac- uous numerical bounds on generalization error for stochastic two-layer two-hidden-unit neural networks via a sensitivity analysis. By optimiz- ing the PAC-Bayes bound directly, we are able to extend their approach and obtain nonvacu- ous generalization bounds for deep stochastic neural network classifiers with millions of pa- rameters trained on only tens of thousands of examples. We connect our findings to recent and old work on flat minima and MDL-based explanations of generalization.

Importance Sampling for Fair Policy Selection

Fri, 11 Aug 2017 00:00:00 +0000

We consider the problem of off-policy policy selection in reinforcement learning: using historical data generated from running one policy to compare two or more policies. We show that approaches based on importance sampling can be unfair—they can select the worse of the two policies more often than not. We give two examples where the unfairness of importance sampling could be practically concerning. We then present sufficient conditions to theoretically guarantee fairness and a related notion of safety. Finally, we provide a practical importance sampling-based estimator to help mitigate one of the systematic sources of unfairness resulting from using importance sampling for policy selection.

Interpreting Lion Behaviour with Nonparametric Probabilistic Programs

Fri, 11 Aug 2017 00:00:00 +0000

We consider the problem of unsupervised learning of meaningful behavioural segments of high-dimensional time-series observations, collected from a pride of African lions1. We demonstrate, by way of a probabilistic programming system (PPS), a methodology which allows for quick iteration over mod- els and Bayesian inferences, which enables us to learn meaningful behavioural segments. We introduce a new Bayesian nonparametric (BNP) state-space model, which extends the hierarchical Dirichlet process (HDP) hidden Markov model (HMM) with an explicit BNP treatment of duration distributions, to deal with different levels of granularity of the latent be- havioural space of the lions. The ease with which this is done exemplifies the flexibility that a PPS gives a scientist2. Furthermore, we combine this approach with unsupervised fea- ture learning, using variational autoencoders.

Value Directed Exploration in Multi-Armed Bandits with Structured Priors

Fri, 11 Aug 2017 00:00:00 +0000

Multi-armed bandits are a quintessential ma- chine learning problem requiring balancing ex- ploration with exploitation. While there has been progress in developing algorithms with strong theoretical guarantees, there has been less focus on practical near-optimal finite-time performance. In this paper, we propose an al- gorithm for Bayesian multi-armed bandits that utilizes approximate value functions. Build- ing on previous work on UCB and Gittins index, we introduce linearly-separable value functions that capture the benefit of exploration when choosing the next arm to pull. Our al- gorithm enjoys a sub-linear performance guar- antee and our simulation results confirm its strength in problems with structured priors. The simplicity and generality of our approach makes it a strong candidate for use in more complex multi-armed bandit problems.

Approximation Complexity of Maximum A Posteriori Inference in Sum-Product Networks

Fri, 11 Aug 2017 00:00:00 +0000

We discuss the computational complexity of approximating maximum a posteriori infer- ence in sum-product networks. We first show NP-hardness in trees of height two by a reduc- tion from maximum independent set; this im- plies non-approximability within a sublinear factor. We show that this is a tight bound, as we can find an approximation within a linear factor in networks of height two. We then show that, in trees of height three, it is NP-hard to ap- proximate the problem within a factor 2f(n) for any sublinear function f of the size of the input n. Again, this bound is tight, as we prove that the usual max-product algorithm finds (in any network) approximations within factor 2c\cdotn for some constant c < 1. Last, we present a sim- ple algorithm, and show that it provably pro- duces solutions at least as good as, and poten- tially much better than, the max-product algo- rithm. We empirically analyze the proposed algorithm against max-product using synthetic and real-world data.

Submodular Variational Inference for Network Reconstruction

Fri, 11 Aug 2017 00:00:00 +0000

In real-world and online social networks, in- dividuals receive and transmit information in real time. Cascading information transmis- sions (e.g. phone calls, text messages, social media posts) may be understood as a realiza- tion of a diffusion process operating on the net- work. The process only traverses and thereby reveals a limited portion of the edges. The network reconstruction/inference problem is to estimate the unrevealed connections. Most existing approaches derive a likelihood and attempt to find the network topology maxi- mizing the likelihood, yielding a highly in- tractable problem. In this paper, we focus on the network reconstruction problem for a broad class of real-world diffusion processes, exem- plified by a network diffusion scheme called respondent-driven sampling (RDS). We prove that under realistic and general models of net- work diffusion, the posterior distribution of an observed RDS realization is a Bayesian log- submodular model. We then propose VINE, a novel, accurate, and computationally efficient variational inference algorithm, for the net- work reconstruction problem under this model. Crucially, we do not assume any particular probabilistic model for the underlying net- work. VINE recovers any connected graph with high accuracy as shown by our experimental results on real-life networks.

FROSH: FasteR Online Sketching Hashing

Fri, 11 Aug 2017 00:00:00 +0000

Many hashing methods, especially those that are in the data-dependent category with good learning accuracy, are still inefficient when dealing with three critical problems in mod- ern data analysis. First, data usually come in a streaming fashion, but most of the exist- ing hashing methods are batch-based models. Second, when data become huge, the exten- sive computational time, large space require- ment, and multiple passes to load the data into memory will be prohibitive. Third, data often lack sufficient label information. Although the recently proposed Online Sketching Hashing (OSH) is promising to alleviate all three issues mentioned above, its training procedure still suffers from a high time complexity. In this paper, we propose a FasteR Online Sketching Hashing (FROSH) method to make the train- ing process faster. Compared with OSH, we leverage fast transform to sketch data more compactly. Particularly, we derive indepen- dent transformations to guarantee the sketch- ing accuracy, and design a novel implemen- tation to make such transformations applica- ble to online data sketching without increas- ing the space cost. We rigorously prove that our method can yield a comparable learning accuracy with a lower time complexity and an equal space cost compared with OSH. Finally, extensive experiments on synthetic and real- world datasets demonstrate the excellent per- formance of our method.

Efficient Online Learning for Optimizing Value of Information: Theory and Application to Interactive Troubleshooting

Fri, 11 Aug 2017 00:00:00 +0000

We consider the optimal value of information problem, where the goal is to sequentially select a set of tests with a minimal cost, so that one can efficiently make the best decision based on the observed outcomes. Existing algorithms are either heuristics with no guar- antees, or scale poorly (with exponential run time in terms of the number of available tests). Moreover, these methods assume a known distribution over the test outcomes, which is often not the case in practice. We propose a sampling-based online learning framework to address the above issues. First, assuming the distribution over hypotheses is known, we propose a dynamic hypoth- esis enumeration strategy, which allows efficient information gathering with strong theoretical guarantees. We show that with sufficient amount of samples, one can identify a near-optimal decision with high proba- bility. Second, when the parameters of the hypotheses distribution are unknown, we propose an algorithm which learns the pa- rameters progressively via posterior sampling in an online fashion. We further establish a rigorous bound on the expected regret. We demonstrate the effectiveness of our approach on a real-world interactive troubleshooting application, and show that one can efficiently make high-quality decisions with low cost.

Analysis of Thompson Sampling for Stochastic Sleeping Bandits

Fri, 11 Aug 2017 00:00:00 +0000

We study a variant of the stochastic multi- armed bandit problem where the set of avail- able arms varies arbitrarily with time (also known as the sleeping bandit problem). We focus on the Thompson Sampling algorithm and consider a regret notion defined with re- spect to the best available arm. Our main re- sult is an O(log T) regret bound for Thompson Sampling, which generalizes a similar bound known for this algorithm from the classical bandit setting. Our bound also matches (up to constants) the best-known lower bound for the sleeping bandit problem. We show via simu- lations that Thompson Sampling outperforms the UCB-style AUER algorithm for the sleep- ing bandit problem.

Robust Model Equivalence using Stochastic Bisimulation for N-Agent Interactive DIDs

Fri, 11 Aug 2017 00:00:00 +0000

I-DIDs suffer disproportionately from the curse of dimensionality dominated by the exponential growth in the number of models over time. Previ- ous methods for scaling I-DIDs identify notions of equivalence between models, such as behav- ioral equivalence (BE). But, this requires that the models be solved first. Also, model space com- pression across agents has not been previously investigated. We present a way to compress the space of models across agents, possibly with dif- ferent frames, and do so without having to solve them first, using stochastic bisimulation. We test our approach on two non-cooperative partially observable domains with up to 20 agents.

On the Complexity of Nash Equilibrium Reoptimization

Fri, 11 Aug 2017 00:00:00 +0000

We provide, to the best of our knowledge, the first study about reoptimization complexity of game-theoretical solutions. In a reoptimization problem, we are given an instance, its optimal solution, and a local modification, and we are asked to find the exact or an approximate so- lution to the modified instance. Reoptimiza- tion is crucial whenever an instance needs to be solved repeatedly and, at each repetition, its parameters may slightly change. In this paper, we focus on Nash equilibrium, being the cen- tral game-theoretical solution. We study the reoptimization of Nash equilibria satisfying some properties (i.e., maximizing/minimizing the social welfare, the utility of a player or the support size) for some different local modifica- tions of the game (i.e., modification of a pay- off or addition/removal of an action), show- ing that such problems are NP-hard. Further- more, we assess the approximation complex- ity of the aforementioned problems, showing that it matches the complexity of the origi- nal (non-reoptimization) problems. Finally, we show that, when finding a Nash equilibrium is thought as an optimization problem, reopti- mization is useful for finding approximate so- lutions. Specifically, it allows one to find $\epsilon$- Nash equilibria with smaller $\epsilon$ than that of the solutions returned by the best known approxi- mation algorithms.

Best Student Paper

Why Rules are Complex: Real-Valued Probabilistic Logic Programs are not Fully Expressive

Fri, 11 Aug 2017 00:00:00 +0000

This paper explores what can and cannot be represented by probabilistic logic programs (PLPs). Propositional PLPs can represent any distribution because they can be acyclic. For relational domains with fixed populations, the probabilistic parameters can be derived as the solutions to polynomial equations. Unfortunately, sometimes they only have complex-valued solutions. Thus PLPs, even with arbitrarily real-valued parameters, cannot represent all distributions. Moreover, they cannot approximate all distributions. Allowing the parameters to be complex numbers, we present a natural truly-cyclic canonical representation that with probability 1 can represent all distributions for (propositional or) relational domains with fixed populations, and, unlike standard representations, has no redundant parameters. Uncertainty in Artificial Intelligence Follow @uai2017 Golden Sponsor

Exact Inference for Relational Graphical Models with Interpreted Functions: Lifted Probabilistic Inference Modulo Theories

Fri, 11 Aug 2017 00:00:00 +0000

Probabilistic Inference Modulo Theories (PIMT) is a recent framework that expands exact infer- ence on graphical models to use richer languages that include arithmetic, equalities, and inequali- ties on both integers and real numbers. In this pa- per, we expand PIMT to a lifted version that also processes random functions and relations. This enhancement is achieved by adapting Inversion, a method from Lifted First-Order Probabilistic Inference literature, to also be modulo theories. This results in the first algorithm for exact proba- bilistic inference that efficiently and simultane- ously exploits random relations and functions, arithmetic, equalities and inequalities.

Regret Minimization Algorithms for the Follower’s Behaviour Identification in Leadership Games

Fri, 11 Aug 2017 00:00:00 +0000

We study for the first time, a leadership game in which one agent, acting as leader, faces another agent, acting as follower, whose behaviour is not known a priori by the leader, being one among a set of possible behavioural profiles. The main motivation is that in real-world applications the common game-theoretical assumption of perfect ratio- nality is rarely met, and any specific assump- tion on bounded rationality models, if wrong, could lead to a significant loss for the leader. The question we pose is whether and how the leader can learn the behavioural profile of a follower in leadership games. This is a “natu- ral” online identification problem: in fact, the leader aims at identifying the follower’s be- havioural profile to exploit at best the poten- tial non-rationality of the opponent, while min- imizing the regret due to the initial lack of in- formation. We propose two algorithms based on different approaches and we provide a re- gret analysis. Furthermore, we experimentally evaluate the pseudo-regret of the algorithms in concrete leadership games, showing that our algorithms outperform the online learning al- gorithms available in the state of the art.

Weighted Model Counting With Function Symbols

Fri, 11 Aug 2017 00:00:00 +0000

Probabilistic relational languages lift the syntax of relational logic for the specification of large-scale probabilistic graphical models, often admitting con- cise descriptions for interacting random variables over classes, hierarchies and constraints. The emer- gence of weighted model counting as an effective and general approach to probabilistic inference has further allowed practitioners to reason about hetero- geneous representations, such as Markov logic net- works and ProbLog programs, by encoding them as a logical theory. However, much of this work has been limited to an essentially propositional setting: the logical model is understood in terms of ground formulas over a fixed and finite domain; no infinite domains, and certainly no function symbols (other than constants). On the one hand, this is not sur- prising, because such features are very problematic from a decidability viewpoint, but on the other, they turn out to be very attractive from the point of view of machine learning applications when there is un- certainty about the existence and identity of objects. In this paper, we reconsider the problem of proba- bilistic reasoning in a logical language with function symbols, and establish some key results that permit effective algorithms.

The Binomial Block Bootstrap Estimator for Evaluating Loss on Dependent Clusters

Fri, 11 Aug 2017 00:00:00 +0000

In this paper, we study the non-IID learn- ing setting where samples exhibit dependency within latent clusters. Our goal is to esti- mate a learner’s loss on new clusters, an ex- tension of the out-of-bag error. Previously developed cross-validation estimators are well suited to the case where the clustering of ob- served data is known a priori. However, as is often the case in real world problems, we are only given a noisy approximation of this clustering, likely the result of some clustering algorithm. This subtle yet potentially signifi- cant issue afflicts domains ranging from image classification to medical diagnostics, where naive cross-validation is an optimistically bi- ased estimator. We present a novel bootstrap technique and corresponding cross-validation method that, somewhat counterintuitively, in- jects additional dependency to asymptotically recover the loss in the independent setting.

Coupling Adaptive Batch Sizes with Learning Rates

Fri, 11 Aug 2017 00:00:00 +0000

Mini-batch stochastic gradient descent and variants thereof have become standard for large-scale empirical risk minimization like the training of neural networks. These meth- ods are usually used with a constant batch size chosen by simple empirical inspection. The batch size significantly influences the behav- ior of the stochastic optimization algorithm, though, since it determines the variance of the gradient estimates. This variance also changes over the optimization process; when using a constant batch size, stability and convergence is thus often enforced by means of a (manually tuned) decreasing learning rate schedule. We propose a practical method for dynamic batch size adaptation. It estimates the vari- ance of the stochastic gradients and adapts the batch size to decrease the variance proportion- ally to the value of the objective function, re- moving the need for the aforementioned learn- ing rate decrease. In contrast to recent related work, our algorithm couples the batch size to the learning rate, directly reflecting the known relationship between the two. On popular im- age classification benchmarks, our batch size adaptation yields faster optimization conver- gence, while simultaneously simplifying learn- ing rate tuning. A TensorFlow implementation is available.