Gaussian Process Regression for Continuous Emotion Recognition with Global Temporal Invariance


Mia Atcheson, Vidhyasaharan Sethu, Julien Epps ;
Proceedings of IJCAI 2017 Workshop on Artificial Intelligence in Affective Computing, PMLR 66:34-44, 2017.


Continuous emotion recognition (CER) is a task which requires the prediction of time series emotional parameter outputs corresponding to query time series inputs given training data in the form of matched pairs of input and output time series. In order to address this task, it is important to be abletomodelnotonly relationshipsbetweenpoints inthe inputandoutput spaces, butalso temporal relationships between points within the output space. Gaussian process regression (GPR) is an inference technique which has desirable properties for CER, including its ability to produce predictive distributions over the outputs rather than only point estimates. However, GPR is generally appliedtopointwisepredictionorinterpolationtasks,ratherthantopredictionsofentirefunctional outputs. We propose a covariance structure that is able to incorporate both input-output and temporal information to produce predictions that take into account the functional nature of CER data. We demonstrate the application of this method to simulated data, and to the AVEC2016 CER task, showing that GPR with this covariance structure is able to make predictions of emotional arousal from audio with over twice the accuracy of a straightforward pointwise application of GPR in the input feature space, and is furthermore able to produce predictions with accuracy approaching that of a competitive CER system using only very general component covariance models.

Related Material