- title: 'Preface' volume: 152 URL: https://proceedings.mlr.press/v152/carlsson21a.html PDF: https://proceedings.mlr.press/v152/carlsson21a/carlsson21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-carlsson21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 1-3 id: carlsson21a issued: date-parts: - 2021 - 9 - 20 firstpage: 1 lastpage: 3 published: 2021-09-20 00:00:00 +0000 - title: 'Approximation to object conditional validity with inductive conformal predictors' abstract: 'Conformal predictors are machine learning algorithms that output prediction sets that have a guarantee of marginal validity for finite samples with minimal distributional assumptions. This is a property that makes conformal predictors useful for machine learning tasks where we require reliable predictions. It would also be desirable to achieve conditional validity in the same setting, in the sense that validity of the prediction intervals remains true regardless of conditioning on any particular property of the object of the prediction. Unfortunately, it has been shown that such conditional validity is impossible to guarantee for non-trivial prediction problems for finite samples. In this article, instead of trying to achieve a strong conditional validity guarantee, an \emph{approximation} to conditional validity is considered and measured empirically. A new algorithm is introduced to do this by iteratively adjusting a conformity measure to deviations from object conditional validity measured in the training data. Experimental results are provided for three data sets that demonstrate (1) in real world machine learning tasks, lack of conditional validity is a measurable problem and (2) that the proposed algorithm is effective at alleviating this problem.' volume: 152 URL: https://proceedings.mlr.press/v152/bellotti21a.html PDF: https://proceedings.mlr.press/v152/bellotti21a/bellotti21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-bellotti21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Anthony family: Bellotti editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 4-23 id: bellotti21a issued: date-parts: - 2021 - 9 - 20 firstpage: 4 lastpage: 23 published: 2021-09-20 00:00:00 +0000 - title: 'Mondrian conformal predictive distributions' abstract: 'The distributions output by a standard (non-normalized) conformal predictive system all have the same shape but differ in location, while a normalized conformal predictive system outputs distributions that differ also in shape, through rescaling. An approach to further increasing the flexibility of the framework is proposed, called \emph{Mondrian conformal predictive distributions}, which are (standard or normalized) conformal predictive distributions formed from multiple Mondrian categories. The effectiveness of the approach is demonstrated with an application to regression forests. By forming categories through binning of the predictions, it is shown that for this model class, the use of Mondrian conformal predictive distributions significantly outperforms the use of both standard and normalized conformal predictive distributions with respect to the continuous- ranked probability score. It is further shown that the use of Mondrian conformal predictive distributions results in as tight prediction intervals as produced by normalized conformal regressors, while improving upon the point predictions of the underlying regression forest.' volume: 152 URL: https://proceedings.mlr.press/v152/bostrom21a.html PDF: https://proceedings.mlr.press/v152/bostrom21a/bostrom21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-bostrom21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Henrik family: Boström - given: Ulf family: Johansson - given: Tuwe family: Löfström editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 24-38 id: bostrom21a issued: date-parts: - 2021 - 9 - 20 firstpage: 24 lastpage: 38 published: 2021-09-20 00:00:00 +0000 - title: 'A lower bound for a prediction algorithm under the Kullback-Leibler game' abstract: 'We obtain a lower bound for an algorithm predicting finite-dimensional distributions (i.e., points from a simplex) under Kullback-Leibler loss. The bound holds w.r.t. the class of softmax linear predictors. We then show that the bound is asymptotically matched by the Bayesian universal algorithm.' volume: 152 URL: https://proceedings.mlr.press/v152/dzhamtyrova21a.html PDF: https://proceedings.mlr.press/v152/dzhamtyrova21a/dzhamtyrova21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-dzhamtyrova21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Raisa family: Dzhamtyrova - given: Yuri family: Kalnishkan editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 39-51 id: dzhamtyrova21a issued: date-parts: - 2021 - 9 - 20 firstpage: 39 lastpage: 51 published: 2021-09-20 00:00:00 +0000 - title: 'Shapley-value based inductive conformal prediction' abstract: 'Shapley values of individual instances were recently proposed for the problem of data valuation. They were defined as the average marginal instance contributions to the performance of a given predictor. In this paper we propose to use Shapley values of individual instances as conformity scores. To compute these values efficiently and exactly we employ a standard algorithm based on nearest neighbor classification and propose a variant of this algorithm for clustered data. Both variants are used for computing Shapley conformity scores for inductive conformal predictors. The experiments show that the Shapley-value conformity scores result in smaller prediction sets for significance level $\epsilon \leq 0.1$ compared with those produced by standard conformity scores (i.e. similarity between true and predicted output values).' volume: 152 URL: https://proceedings.mlr.press/v152/jaramillo21a.html PDF: https://proceedings.mlr.press/v152/jaramillo21a/jaramillo21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-jaramillo21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: William Lopez family: Jaramillo - given: Evgueni family: Smirnov editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 52-71 id: jaramillo21a issued: date-parts: - 2021 - 9 - 20 firstpage: 52 lastpage: 71 published: 2021-09-20 00:00:00 +0000 - title: 'Conformal uncertainty sets for robust optimization' abstract: 'Decision-making under uncertainty is hugely important for any decisions sensitive to perturbations in observed data. One method of incorporating uncertainty into making optimal decisions is through robust optimization, which minimizes the worst-case scenario over some \emph{uncertainty set}. We connect conformal prediction regions to robust optimization, providing finite sample valid and conservative ellipsoidal uncertainty sets, aptly named conformal uncertainty sets. In pursuit of this connection we explicitly define Mahalanobis distance as a potential conformity score in full conformal prediction. We also compare the coverage and optimization performance of conformal uncertainty sets, specifically generated with Mahalanobis distance, to traditional ellipsoidal uncertainty sets on a collection of simulated robust optimization examples.' volume: 152 URL: https://proceedings.mlr.press/v152/johnstone21a.html PDF: https://proceedings.mlr.press/v152/johnstone21a/johnstone21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-johnstone21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Chancellor family: Johnstone - given: Bruce family: Cox editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 72-90 id: johnstone21a issued: date-parts: - 2021 - 9 - 20 firstpage: 72 lastpage: 90 published: 2021-09-20 00:00:00 +0000 - title: 'Synergy conformal prediction' abstract: 'Conformal prediction is a machine learning methodology that produces valid prediction regions. Ensembles of conformal predictors have been proposed to improve the informational efficiency of inductive conformal predictors by combining p-values, however, the validity of such methods has been an open problem. We introduce synergy conformal prediction which is an ensemble method that combines monotonic conformity scores, and is capable of producing valid prediction intervals. We study the applicability in three scenarios; where data is partitioned, where an ensemble of different machine learning methods is used, and where data is unpartitioned. We evaluate the method on 10 data sets and show that the synergy conformal predictor produces valid prediction intervals that on partitioned data performs well compared to the most efficient model trained on individual partitions, making it a viable approach for federated settings when data cannot be pooled. We also show that our method has advantages over current ensembles of conformal predictors by producing valid and efficient results on unpartitioned data, and that it is less computationally demanding.' volume: 152 URL: https://proceedings.mlr.press/v152/gauraha21a.html PDF: https://proceedings.mlr.press/v152/gauraha21a/gauraha21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-gauraha21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Niharika family: Gauraha - given: Ola family: Spjuth editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 91-110 id: gauraha21a issued: date-parts: - 2021 - 9 - 20 firstpage: 91 lastpage: 110 published: 2021-09-20 00:00:00 +0000 - title: 'Calibrating multi-class models' abstract: 'Predictive models communicating algorithmic confidence are very informative, but only if well-calibrated and sharp, i.e., providing accurate probability estimates adjusted for each instance. While almost all machine learning algorithms are able to produce probability estimates, these are often poorly calibrated, thus requiring external calibration. For multiclass problems, external calibration has typically been done using one-vs-all or all-vs-all schemes, thus adding to the computational complexity, but also making it impossible to analyze and inspect the predictive models. In this paper, we suggest a novel approach for calibrating inherently multi-class models. Instead of providing a probability distribution over all labels, the estimation is of the probability that the class label predicted by the underlying model is correct. In an extensive empirical study, it is shown that the suggested approach, when applied to both Platt scaling and Venn-Abers, is able to improve the probability estimates from decision trees, random forests and extreme gradient boosting.' volume: 152 URL: https://proceedings.mlr.press/v152/johansson21a.html PDF: https://proceedings.mlr.press/v152/johansson21a/johansson21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-johansson21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Ulf family: Johansson - given: Tuwe family: Löfström - given: Henrik family: Boström editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 111-130 id: johansson21a issued: date-parts: - 2021 - 9 - 20 firstpage: 111 lastpage: 130 published: 2021-09-20 00:00:00 +0000 - title: 'Conformal testing in a binary model situation' abstract: 'Conformal testing is a way of testing the IID assumption based on conformal prediction. The topic of this paper is experimental evaluation of the performance of conformal testing in a model situation in which IID binary observations generated from a Bernoulli distribution are followed by IID binary observations generated from another Bernoulli distribution, with the parameters of the distributions and changepoint known or unknown. Existing conformal test martingales can be used for this task and work well in simple cases, but their efficiency can be improved greatly.' volume: 152 URL: https://proceedings.mlr.press/v152/vovk21a.html PDF: https://proceedings.mlr.press/v152/vovk21a/vovk21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-vovk21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Vladimir family: Vovk editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 131-150 id: vovk21a issued: date-parts: - 2021 - 9 - 20 firstpage: 131 lastpage: 150 published: 2021-09-20 00:00:00 +0000 - title: 'Impact of model-agnostic nonconformity functions on efficiency of conformal classifiers: an extensive study' abstract: 'The property of conformal predictors to guarantee the required accuracy rate makes this framework attractive in various practical applications. However, this property is achieved at a price of reduction in precision. In the case of conformal classification, the system can output multiple class labels instead of one. It is also known, that the choice of nonconformity function has a major impact on the efficiency of conformal classifiers. Recently, it was shown that different model-agnostic nonconformity functions result in conformal classifiers with different characteristics. For a Neural Network-based conformal classifier, the \emph{inverse probability} (or hinge loss) allows minimizing the average number of predicted labels, and \emph{margin} results in a larger fraction of singleton predictions. In this work, we aim to further extend this study. We perform an experimental evaluation using 8 different classification algorithms and discuss when the previously observed relationship holds or not. Additionally, we propose a successful method to combine the properties of these two nonconformity functions.' volume: 152 URL: https://proceedings.mlr.press/v152/aleksandrova21a.html PDF: https://proceedings.mlr.press/v152/aleksandrova21a/aleksandrova21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-aleksandrova21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Marharyta family: Aleksandrova - given: Oleg family: Chertov editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 151-170 id: aleksandrova21a issued: date-parts: - 2021 - 9 - 20 firstpage: 151 lastpage: 170 published: 2021-09-20 00:00:00 +0000 - title: 'Using inductive conformal martingales for addressing concept drift in data stream classification' abstract: 'In this paper, we investigate the use of Inductive Conformal Martingales (ICM) with the histogram betting function for detecting the occurrence of concept drift (CD) in data stream classification. A change in the data distribution will almost surely affect the performance of our classification model resulting in false predictions. Therefore, a reliable and fast detection of the point at which a CD occurs, allows effective retraining of the model to recover accuracy. Our approach is based on ICM with the histogram betting function, which is much more computationally efficient than alternative ICM approaches. To accelerate the process of detecting CD we also modify the ICM and examine different parameters of the histogram betting function. We evaluate the proposed approach on three benchmark datasets, namely STAGGER, SEA and ELEC, presenting different measures of its performance and comparing it with existing methods in the literature.' volume: 152 URL: https://proceedings.mlr.press/v152/eliades21a.html PDF: https://proceedings.mlr.press/v152/eliades21a/eliades21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-eliades21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Charalambos family: Eliades - given: Harris family: Papadopoulos editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 171-190 id: eliades21a issued: date-parts: - 2021 - 9 - 20 firstpage: 171 lastpage: 190 published: 2021-09-20 00:00:00 +0000 - title: 'Retrain or not retrain: conformal test martingales for change-point detection' abstract: 'We argue for supplementing the process of training a prediction algorithm by setting up a scheme for detecting the moment when the distribution of the data changes and the algorithm needs to be retrained. Our proposed schemes are based on exchangeability martingales, i.e., processes that are martingales under any exchangeable distribution for the data. Our method, based on conformal prediction, is general and can be applied on top of any modern prediction algorithm. Its validity is guaranteed, and in this paper we make first steps in exploring its efficiency.' volume: 152 URL: https://proceedings.mlr.press/v152/vovk21b.html PDF: https://proceedings.mlr.press/v152/vovk21b/vovk21b.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-vovk21b.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Vladimir family: Vovk - given: Ivan family: Petej - given: Ilia family: Nouretdinov - given: Ernst family: Ahlberg - given: Lars family: Carlsson - given: Alex family: Gammerman editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 191-210 id: vovk21b issued: date-parts: - 2021 - 9 - 20 firstpage: 191 lastpage: 210 published: 2021-09-20 00:00:00 +0000 - title: 'Class-wise confidence for debt prediction in real estate management: discussion and lessons learned from an application' abstract: 'The prediction of tenants likely to fall into a debt situation is a key issue for social property owners in real estate. It is even more important for them to limit the number of people falsely predicted to be in debt to avoid incurring unnecessary costs (in time and money), for instance by sending agents to prevent the debt. In this paper, we adapt Mondrian conformal prediction to control the error rate of this class, while keeping a level of confidence chosen by the social property owner, or more generally by the user. We also test this small adaptation with different splitting strategies and discuss the obtained results, those later showing promising results, in the sense that they show that our approach can work, as well as pointing out and discussing difficulties, in the sense that conformal prediction fails on some settings of particular interest to the end-user.' volume: 152 URL: https://proceedings.mlr.press/v152/messoudi21a.html PDF: https://proceedings.mlr.press/v152/messoudi21a/messoudi21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-messoudi21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Soundouss family: Messoudi - given: Sébastien family: Destercke - given: Sylvain family: Rousseau editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 211-228 id: messoudi21a issued: date-parts: - 2021 - 9 - 20 firstpage: 211 lastpage: 228 published: 2021-09-20 00:00:00 +0000 - title: 'Evaluation of updating strategies for conformal predictive systems in the presence of extreme events' abstract: 'Six different strategies for updating split conformal predictive systems in an online (streaming) setting are evaluated. The updating strategies vary in the extent and frequency of retraining as well as in how training data is split into proper training and calibration sets. An empirical evaluation is presented, considering passenger booking data from a ferry company, which stretches over a number of years. The passenger volumes have changed drastically during 2020 due to COVID-19 and part of the evaluation is focusing on which updating strategies work best under such circumstances. Some strategies are observed to outperform others with respect to continuous ranked probability score and validity, highlighting the potential value of choosing a proper strategy.' volume: 152 URL: https://proceedings.mlr.press/v152/werner21a.html PDF: https://proceedings.mlr.press/v152/werner21a/werner21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-werner21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Hugo family: Werner - given: Lars family: Carlsson - given: Ernst family: Ahlberg - given: Henrik family: Boström editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 229-242 id: werner21a issued: date-parts: - 2021 - 9 - 20 firstpage: 229 lastpage: 242 published: 2021-09-20 00:00:00 +0000 - title: 'Transformer-based conformal predictors for paraphrase detection' abstract: 'Transformer architectures have established themselves as the state-of-the-art in many areas of natural language processing (NLP), including paraphrase detection (PD). However, they do not include a confidence estimation for each prediction and, in many cases, the applied models are poorly calibrated. These features are essential for numerous real-world applications. For example, in those cases when PD is used for sensitive tasks, like plagiarism detection, hate speech recognition or in medical NLP, mistakes might be very costly. In this work we build several variants of transformer- based conformal predictors and study their behaviour on a standard PD dataset. We show that our models are able to produce \emph{valid} predictions while retaining the accuracy of the original transformer-based models. The proposed technique can be extended to many more NLP problems that are currently being investigated.' volume: 152 URL: https://proceedings.mlr.press/v152/giovannotti21a.html PDF: https://proceedings.mlr.press/v152/giovannotti21a/giovannotti21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-giovannotti21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Patrizio family: Giovannotti - given: Alex family: Gammerman editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 243-265 id: giovannotti21a issued: date-parts: - 2021 - 9 - 20 firstpage: 243 lastpage: 265 published: 2021-09-20 00:00:00 +0000 - title: 'A non-conformity approach towards post-prostatectomy metastasis estimation using a multicentre prostate cancer database' abstract: 'Prostate cancer is among the most common type of cancer in men worldwide. Despite the use of clinical indicators, as part of simple rule-based strategies, stratifying patients diagnosed with prostate cancer into risk groups to reliably reflect oncological prognosis remains challenging. Machine Learning (ML) offers the possibility to develop estimation models based on routinely evaluated patient or tumor characteristics. In the present study, the estimation of metastasis in prostate patients after primary treatments (radical prostatectomy) with the aid of Support Vector Machines (SVMs) and Conformal Predictors (CP) was evaluated. We show that the use of ML models can complement classical statistical approaches. Moreover, the application of CP, on top of an underlying ML model, renders a probabilistic outcome that combines the simplicity of a clinical indicator with the precision of a ML approach. The TriNetX Research Network, an electronic health records database with datasets from several United States health care organizations, was used in this study. This approach can be further adapted to support clinical decision making in prostate and other types of cancer.' volume: 152 URL: https://proceedings.mlr.press/v152/chatzichristos21a.html PDF: https://proceedings.mlr.press/v152/chatzichristos21a/chatzichristos21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-chatzichristos21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Christos family: Chatzichristos - given: Jose-Felipe family: Golib-Dzib - given: Andries family: Clinckaert - given: Wouter family: Everaerts - given: Maarten family: De Vos - given: Martine family: Lewi editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 266-285 id: chatzichristos21a issued: date-parts: - 2021 - 9 - 20 firstpage: 266 lastpage: 285 published: 2021-09-20 00:00:00 +0000 - title: 'Conformal prediction and its integration within visual analytics toolbox' abstract: 'Conformal prediction is a machine learning approach to report on the reliability of predictive models when applied to new cases. Machine learning techniques are gaining in complexity, and assessing their reliability may be an essential part of explaining the inner workings of predictive models. For practical purposes and dissemination of conformal prediction techniques, we must include these within easily accessible toolboxes. In machine learning, a significant subset of such toolboxes is those that use work flows and visual programming. Here, we report on an example of such a toolbox, Python implementation of conformal prediction library, and our initial efforts and ideas to democratize conformal prediction.' volume: 152 URL: https://proceedings.mlr.press/v152/hocevar21a.html PDF: https://proceedings.mlr.press/v152/hocevar21a/hocevar21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-hocevar21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Tomaž family: Hočevar - given: Blaž family: Zupan editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 286-293 id: hocevar21a issued: date-parts: - 2021 - 9 - 20 firstpage: 286 lastpage: 293 published: 2021-09-20 00:00:00 +0000 - title: 'Confidence machine learning for cutting tool life prediction' abstract: 'The work aims to develop an automatic cutting tool life prediction model for die-cuts machine at Parafix. Such model will be able to estimate how long a given tool is likely to last, in order to improve performance and productivity. This work is part of the KTP project between Parafix and University of Brighton.' volume: 152 URL: https://proceedings.mlr.press/v152/wilson21a.html PDF: https://proceedings.mlr.press/v152/wilson21a/wilson21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-wilson21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Nishant family: Wilson - given: Steve family: Barwick - given: Vince family: Booker - given: Tom family: Mildenhall - given: Laura family: Still - given: Yan family: Wang - given: Khuong family: An Nguyen editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 294-296 id: wilson21a issued: date-parts: - 2021 - 9 - 20 firstpage: 294 lastpage: 296 published: 2021-09-20 00:00:00 +0000 - title: 'Protected probabilistic classification' abstract: 'This poster proposes a way of protecting algorithms for probabilistic binary classification against changes in the data distribution.' volume: 152 URL: https://proceedings.mlr.press/v152/vovk21c.html PDF: https://proceedings.mlr.press/v152/vovk21c/vovk21c.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-vovk21c.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Vladimir family: Vovk - given: Ivan family: Petej - given: Alex family: Gammerman editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 297-299 id: vovk21c issued: date-parts: - 2021 - 9 - 20 firstpage: 297 lastpage: 299 published: 2021-09-20 00:00:00 +0000 - title: 'Conformal changepoint detection in continuous model situations' abstract: 'Conformal prediction provides a way of testing the IID assumption, which is the standard assumption in machine learning. A natural question is whether this way of testing is efficient. A typical situation where the IID assumption is broken is the existence of a changepoint at which the distribution of the data changes. We study the case of a change from one continuous distribution to another with both distributions belonging to standard parametric families. Our conclusion is that the conformal approach to testing the IID assumption is efficient, at least to some degree.' volume: 152 URL: https://proceedings.mlr.press/v152/nouretdinov21a.html PDF: https://proceedings.mlr.press/v152/nouretdinov21a/nouretdinov21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-nouretdinov21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Ilia family: Nouretdinov - given: Vladimir family: Vovk - given: Alex family: Gammerman editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 300-302 id: nouretdinov21a issued: date-parts: - 2021 - 9 - 20 firstpage: 300 lastpage: 302 published: 2021-09-20 00:00:00 +0000 - title: 'Fast conformal classification using influence functions' abstract: 'We use influence functions from robust statistics to speed up full conformal prediction. Traditionally, conformal prediction requires retraining multiple leave-one-out classifiers to calculate p-values for each test point. By using influence functions, we are able to approximate this procedure and to speed up considerably the time complexity.' volume: 152 URL: https://proceedings.mlr.press/v152/bhatt21a.html PDF: https://proceedings.mlr.press/v152/bhatt21a/bhatt21a.pdf edit: https://github.com/mlresearch//v152/edit/gh-pages/_posts/2021-09-20-bhatt21a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications' publisher: 'PMLR' author: - given: Umang family: Bhatt - given: Adrian family: Weller - given: Giovanni family: Cherubin editor: - given: Lars family: Carlsson - given: Zhiyuan family: Luo - given: Giovanni family: Cherubin - given: Khuong family: An Nguyen page: 303-305 id: bhatt21a issued: date-parts: - 2021 - 9 - 20 firstpage: 303 lastpage: 305 published: 2021-09-20 00:00:00 +0000