Deep Ensembles for Inter-Domain Arousal Recognition

Martin Gjoreski, Hristijian Gjoreski, Mitja Lustrek, Matjaz Gams
Proceedings of IJCAI 2018 2nd Workshop on Artificial Intelligence in Affective Computing, PMLR 86:52-64, 2020.

Abstract

The computer science field Affective Computing, which studies and develops emotional intelligent systems, has been active for almost two decades now with limited results. Arousal as the dimension that represents the intensity of the emotions, represents similar recognition problems. This is the first study that analyzes six publicly available datasets for arousal recognition from physiological signals and proposes a method capable of combining them. The novel method, an inter-domain Deep Neural Network (DNN) ensemble, is compared to classical machine learning (ML). For both methods, the raw data from Galvanic Skin Response (GSR), Electrocardiography ECG, and Blood Volume Pulse (BVP) sensors is processed and transformed into a common spectro-temporal space of R-R intervals and GSR data. For the classical ML algorithms, features are extracted, and for the DNN algorithms, two different approaches were taken: a fully connected DNN trained with the same features as the classical ML algorithms (DNN-Features) and a Convolutional Neural Network (CNN) trained with the temporal representation of the GSR signal (CNN-GSR). Finally, a fully connected DNN meta learner is trained to utilize the knowledge from the two different DNNs and to tune the DNN models for the target dataset. The experimental results showed that the novel DNN ensemble method outperforms the classical ML methods and the non-ensemble DNN methods. Additionally, the CNN-GSR model learned that the peaks of the GSR signal contain the most information regarding the arousal, thus the network developed filters to emphasize those parts.

Cite this Paper

BibTeX
@InProceedings{pmlr-v86-gjoreski20a, title = {Deep Ensembles for Inter-Domain Arousal Recognition}, author = {Gjoreski, Martin and Gjoreski, Hristijian and Lustrek, Mitja and Gams, Matjaz}, booktitle = {Proceedings of IJCAI 2018 2nd Workshop on Artificial Intelligence in Affective Computing}, pages = {52--64}, year = {2020}, editor = {Hsu, William and Yates, Heath}, volume = {86}, series = {Proceedings of Machine Learning Research}, month = {15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v86/gjoreski20a/gjoreski20a.pdf}, url = {http://proceedings.mlr.press/v86/gjoreski20a.html}, abstract = {The computer science field Affective Computing, which studies and develops emotional intelligent systems, has been active for almost two decades now with limited results. Arousal as the dimension that represents the intensity of the emotions, represents similar recognition problems. This is the first study that analyzes six publicly available datasets for arousal recognition from physiological signals and proposes a method capable of combining them. The novel method, an inter-domain Deep Neural Network (DNN) ensemble, is compared to classical machine learning (ML). For both methods, the raw data from Galvanic Skin Response (GSR), Electrocardiography ECG, and Blood Volume Pulse (BVP) sensors is processed and transformed into a common spectro-temporal space of R-R intervals and GSR data. For the classical ML algorithms, features are extracted, and for the DNN algorithms, two different approaches were taken: a fully connected DNN trained with the same features as the classical ML algorithms (DNN-Features) and a Convolutional Neural Network (CNN) trained with the temporal representation of the GSR signal (CNN-GSR). Finally, a fully connected DNN meta learner is trained to utilize the knowledge from the two different DNNs and to tune the DNN models for the target dataset. The experimental results showed that the novel DNN ensemble method outperforms the classical ML methods and the non-ensemble DNN methods. Additionally, the CNN-GSR model learned that the peaks of the GSR signal contain the most information regarding the arousal, thus the network developed filters to emphasize those parts.} }
Endnote
%0 Conference Paper %T Deep Ensembles for Inter-Domain Arousal Recognition %A Martin Gjoreski %A Hristijian Gjoreski %A Mitja Lustrek %A Matjaz Gams %B Proceedings of IJCAI 2018 2nd Workshop on Artificial Intelligence in Affective Computing %C Proceedings of Machine Learning Research %D 2020 %E William Hsu %E Heath Yates %F pmlr-v86-gjoreski20a %I PMLR %P 52--64 %U http://proceedings.mlr.press/v86/gjoreski20a.html %V 86 %X The computer science field Affective Computing, which studies and develops emotional intelligent systems, has been active for almost two decades now with limited results. Arousal as the dimension that represents the intensity of the emotions, represents similar recognition problems. This is the first study that analyzes six publicly available datasets for arousal recognition from physiological signals and proposes a method capable of combining them. The novel method, an inter-domain Deep Neural Network (DNN) ensemble, is compared to classical machine learning (ML). For both methods, the raw data from Galvanic Skin Response (GSR), Electrocardiography ECG, and Blood Volume Pulse (BVP) sensors is processed and transformed into a common spectro-temporal space of R-R intervals and GSR data. For the classical ML algorithms, features are extracted, and for the DNN algorithms, two different approaches were taken: a fully connected DNN trained with the same features as the classical ML algorithms (DNN-Features) and a Convolutional Neural Network (CNN) trained with the temporal representation of the GSR signal (CNN-GSR). Finally, a fully connected DNN meta learner is trained to utilize the knowledge from the two different DNNs and to tune the DNN models for the target dataset. The experimental results showed that the novel DNN ensemble method outperforms the classical ML methods and the non-ensemble DNN methods. Additionally, the CNN-GSR model learned that the peaks of the GSR signal contain the most information regarding the arousal, thus the network developed filters to emphasize those parts.
APA
Gjoreski, M., Gjoreski, H., Lustrek, M. & Gams, M.. (2020). Deep Ensembles for Inter-Domain Arousal Recognition. Proceedings of IJCAI 2018 2nd Workshop on Artificial Intelligence in Affective Computing, in Proceedings of Machine Learning Research 86:52-64 Available from http://proceedings.mlr.press/v86/gjoreski20a.html.

Related Material