A Deep Neural Network Conformal Predictor for Multilabel Text Classification
[edit]
Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, PMLR 105:228245, 2019.
Abstract
We investigate the use of inductive conformal prediction (ICP)
for the task of multilabel text classification
and present preliminary experimental results for a subset of the original Reuters21578 dataset.
Our underlying classification model is a deep neural network configuration
which consists of a trainable embedding layer, a convolutional layer and two dense feedforward layers,
arranged sequentially,
with sigmoid outputs representing the individual unique labels of the selected subset.
Following the powerset approach,
we assign nonconformity scores to labelsets
from which the corresponding pvalues and predictionsets are determined
and we experiment with a number of different versions of a nonconformity measure.
Our results indicate a good performance for the underlying model
which is carried on to the ICP without any significant accuracy loss
and with the added benefits of predictionspecific confidence information.
Predictionsets are tight enough to be practically useful
even though the multilabel subset contains tens of thousands of possible label combinations
and empirical errorrates confirm that our outputs are wellcalibrated.
Related Material


