Proceedings of IJCAI 2018 2nd Workshop on Artificial Intelligence in Affective Computing on 15 July 2018 Published as Volume 86 by the Proceedings of Machine Learning Research on 27 April 2020. Volume Edited by: William Hsu Heath Yates Series Editors: Neil D. Lawrence Mark Reid http://proceedings.mlr.press/v86/ Sun, 05 Jul 2026 15:12:06 +0000 Sun, 05 Jul 2026 15:12:06 +0000 Jekyll v3.10.0 Affective experience prediction using different data modalities measured from an individual such as their facial expression or physiological signals has received substantial research attention in recent years. However, most studies ignore the fact that people besides having different responses under affective stimuli, may also have different resting dynamics (embedded in both facial and physiological patterns) to begin with. In this paper, we present a multimodal approach to simultaneously analyze facial movements and several peripheral physiological signals to decode individualized affective experiences under positive and negative emotional contexts, while considering their personalized resting dynamics. We propose a person-specific recurrence network to quantify the dynamics present in the person’s facial movements and physiological data. Facial movement is represented using a robust head vs. 3D face landmark localization and tracking approach, and physiological data are processed by extracting known attributes related to the underlying affective experience. The dynamical coupling between different input modalities is then assessed through the extraction of several complex recurrent network metrics. Inference models are then trained using these metrics as features to predict individual’s affective experience in a given context, after their resting dynamics are excluded from their response. We validated our approach using a multimodal dataset consists of (i) facial videos and (ii) several peripheral physiological signals, synchronously recorded from 12 participants while watching 4 emotion-eliciting video-based stimuli. The affective experience prediction results signified that our multimodal fusion method improves the prediction accuracy up to 19% when compared to the prediction using only one or a subset of the input modalities. Furthermore, we gained prediction improvement for affective experience by considering the effect of individualized resting dynamics. Mon, 27 Apr 2020 00:00:00 +0000 http://proceedings.mlr.press/v86/yin20a.html http://proceedings.mlr.press/v86/yin20a.html The goal of this paper is to develop a methodology and model to classify and characterize the arousal state of participants in a built environment. Demonstrating this showcases the potential of developing an intelligent system capable of both classifying and predicting biometric arousal state. This classification process is traditionally performed by human experts. Our approach can be leveraged to take advantage of the diversity of real-time sensor data to inform the development of smart(er) environments to improve human health. Mon, 27 Apr 2020 00:00:00 +0000 http://proceedings.mlr.press/v86/yates20a.html http://proceedings.mlr.press/v86/yates20a.html The question of representing emotion computationally remains largely unanswered: popular approaches require annotators to assign a magnitude (or a class) of some emotional dimension, while an alternative is to focus on the relationship between two or more options. Recent evidence in affective computing suggests that following a methodology of ordinal annotations and processing leads to better reliability and validity of the model. This paper compares the generality of classification methods versus preference learning methods in predicting the levels of arousal in two widely used affective datasets. Findings of this initial study further validate the hypothesis that approaching affect labels as ordinal data and building models via preference learning yields models of better validity. Mon, 27 Apr 2020 00:00:00 +0000 http://proceedings.mlr.press/v86/melhart20a.html http://proceedings.mlr.press/v86/melhart20a.html A known deficit of modern machine learning (ML) and deep learning (DL) methodology is that models must be carefully fine-tuned in order to solve a particular task. Most algorithms cannot generalize well to even highly similar tasks, let alone exhibit signs of artificial general intelligence (AGI). To address this problem, researchers have explored developing loss functions that act as intrinsic motivators that could drive an ML or DL agent to learn across a number of domains. This paper argues that an important and useful intrinsic motivator is that of social interaction. We posit that making an AI agent aware of implicit social feedback from humans can allow for faster learning of more generalizable and useful representations, and could potentially impact AI safety. We collect social feedback in the form of facial expression reactions to samples from Sketch RNN, an LSTM-based variational autoencoder (VAE) designed to produce sketch drawings. We use a Latent Constraints GAN (LC-GAN) to learn from the facial feedback of a small group of viewers, by optimizing the model to produce sketches that it predicts will lead to more positive facial expressions. We show in multiple independent evaluations that the model trained with facial feedback produced sketches that are more highly rated, and induce significantly more positive facial expressions. Thus, we establish that implicit social feedback can improve the output of a deep learning model. Mon, 27 Apr 2020 00:00:00 +0000 http://proceedings.mlr.press/v86/jaques20a.html http://proceedings.mlr.press/v86/jaques20a.html The computer science field Affective Computing, which studies and develops emotional intelligent systems, has been active for almost two decades now with limited results. Arousal as the dimension that represents the intensity of the emotions, represents similar recognition problems. This is the first study that analyzes six publicly available datasets for arousal recognition from physiological signals and proposes a method capable of combining them. The novel method, an inter-domain Deep Neural Network (DNN) ensemble, is compared to classical machine learning (ML). For both methods, the raw data from Galvanic Skin Response (GSR), Electrocardiography ECG, and Blood Volume Pulse (BVP) sensors is processed and transformed into a common spectro-temporal space of R-R intervals and GSR data. For the classical ML algorithms, features are extracted, and for the DNN algorithms, two different approaches were taken: a fully connected DNN trained with the same features as the classical ML algorithms (DNN-Features) and a Convolutional Neural Network (CNN) trained with the temporal representation of the GSR signal (CNN-GSR). Finally, a fully connected DNN meta learner is trained to utilize the knowledge from the two different DNNs and to tune the DNN models for the target dataset. The experimental results showed that the novel DNN ensemble method outperforms the classical ML methods and the non-ensemble DNN methods. Additionally, the CNN-GSR model learned that the peaks of the GSR signal contain the most information regarding the arousal, thus the network developed filters to emphasize those parts. Mon, 27 Apr 2020 00:00:00 +0000 http://proceedings.mlr.press/v86/gjoreski20a.html http://proceedings.mlr.press/v86/gjoreski20a.html