Fast Adaptation of Deep Models for Facial Action Unit Detection Using Model-Agnostic Meta-Learning

Mihee Lee, Ognjen Rudovic, Vladimir Pavlovic, Maja Pantic
Proceedings of IJCAI 2019 3rd Workshop on Artificial Intelligence in Affective Computing, PMLR 122:9-27, 2020.

Abstract

Detecting facial action unit (AU) activations is one of the key steps in automatic recognition of facial expressions of human emotion and cognitive states. While there are different approaches proposed for this task, most of these are trained only for a specific (sub)set of AUs. As such, they cannot easily adapt to the task of detection of new AUs which are not initially used to train the target models. In this paper, we propose a deep learning approach for facial AU detection that can adapt to a new AU and/or target subject by leveraging only a few labeled samples from the new task (either an AU or subject). We use the notion of the model-agnostic meta-learning, originally proposed for the general image recognition/detection tasks, to design our deep learning models for AU detection. Specifically, each subject and/or AU is treated as a new learning task and the model learns to adapt based on the knowledge of the previously seen tasks. We show on two benchmark datasets (BP4D and DISFA) for facial AU detection that the proposed approach can easily be adapted to new tasks. By using as few as one or five labeled examples from the target task, our approach achieves large improvements over the baseline (non-adapted) deep models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v122-lee20a, title = {Fast Adaptation of Deep Models for Facial Action Unit Detection Using Model-Agnostic Meta-Learning}, author = {Lee, Mihee and Rudovic, Ognjen and Pavlovic, Vladimir and Pantic, Maja}, booktitle = {Proceedings of IJCAI 2019 3rd Workshop on Artificial Intelligence in Affective Computing}, pages = {9--27}, year = {2020}, editor = {Hsu, William}, volume = {122}, series = {Proceedings of Machine Learning Research}, month = {10 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v122/lee20a/lee20a.pdf}, url = {https://proceedings.mlr.press/v122/lee20a.html}, abstract = {Detecting facial action unit (AU) activations is one of the key steps in automatic recognition of facial expressions of human emotion and cognitive states. While there are different approaches proposed for this task, most of these are trained only for a specific (sub)set of AUs. As such, they cannot easily adapt to the task of detection of new AUs which are not initially used to train the target models. In this paper, we propose a deep learning approach for facial AU detection that can adapt to a new AU and/or target subject by leveraging only a few labeled samples from the new task (either an AU or subject). We use the notion of the model-agnostic meta-learning, originally proposed for the general image recognition/detection tasks, to design our deep learning models for AU detection. Specifically, each subject and/or AU is treated as a new learning task and the model learns to adapt based on the knowledge of the previously seen tasks. We show on two benchmark datasets (BP4D and DISFA) for facial AU detection that the proposed approach can easily be adapted to new tasks. By using as few as one or five labeled examples from the target task, our approach achieves large improvements over the baseline (non-adapted) deep models. } }
Endnote
%0 Conference Paper %T Fast Adaptation of Deep Models for Facial Action Unit Detection Using Model-Agnostic Meta-Learning %A Mihee Lee %A Ognjen Rudovic %A Vladimir Pavlovic %A Maja Pantic %B Proceedings of IJCAI 2019 3rd Workshop on Artificial Intelligence in Affective Computing %C Proceedings of Machine Learning Research %D 2020 %E William Hsu %F pmlr-v122-lee20a %I PMLR %P 9--27 %U https://proceedings.mlr.press/v122/lee20a.html %V 122 %X Detecting facial action unit (AU) activations is one of the key steps in automatic recognition of facial expressions of human emotion and cognitive states. While there are different approaches proposed for this task, most of these are trained only for a specific (sub)set of AUs. As such, they cannot easily adapt to the task of detection of new AUs which are not initially used to train the target models. In this paper, we propose a deep learning approach for facial AU detection that can adapt to a new AU and/or target subject by leveraging only a few labeled samples from the new task (either an AU or subject). We use the notion of the model-agnostic meta-learning, originally proposed for the general image recognition/detection tasks, to design our deep learning models for AU detection. Specifically, each subject and/or AU is treated as a new learning task and the model learns to adapt based on the knowledge of the previously seen tasks. We show on two benchmark datasets (BP4D and DISFA) for facial AU detection that the proposed approach can easily be adapted to new tasks. By using as few as one or five labeled examples from the target task, our approach achieves large improvements over the baseline (non-adapted) deep models.
APA
Lee, M., Rudovic, O., Pavlovic, V. & Pantic, M.. (2020). Fast Adaptation of Deep Models for Facial Action Unit Detection Using Model-Agnostic Meta-Learning. Proceedings of IJCAI 2019 3rd Workshop on Artificial Intelligence in Affective Computing, in Proceedings of Machine Learning Research 122:9-27 Available from https://proceedings.mlr.press/v122/lee20a.html.

Related Material