Development of an Enhanced Machine Learning Model for Deception Detection Leveraging Facial Action Units and Linguistic Features

Adetoye Oluwatoyin Adedokun, Adebola K. Ojo
Proceedings of IndabaX Nigeria 2026: Building Scalable AI That Works: From Research to Deployment in Resource-Constrained Environments, PMLR 319:62-73, 2026.

Abstract

This study proposes a bimodal machine learning approach for multiclass deception detection by integrating facial Action Unit (AU) features and linguistic features extracted from a real-life dataset of video interviews of suspected criminal perpetrators, persons of interest, and convicted criminals. Facial features were obtained using the Facial Action Coding System (FACS), while linguistic cues were derived using Linguistic Inquiry Word Count (LIWC) scores. A mid-level data fusion strategy combines the extracted features into a unified representation. A Random Forest classifier applied to 3,720 real-life samples with 80:20 split and 10-fold cross-validation achieved an overall classification accuracy of 88%. Results confirm that combining facial and linguistic cues from real-life datasets provides a richer representation of deceptive behaviour.

Cite this Paper

BibTeX
@InProceedings{pmlr-v319-adedokun26a, title = {Development of an Enhanced Machine Learning Model for Deception Detection Leveraging Facial Action Units and Linguistic Features}, author = {Adedokun, Adetoye Oluwatoyin and Ojo, Adebola K.}, booktitle = {Proceedings of IndabaX Nigeria 2026: Building Scalable AI That Works: From Research to Deployment in Resource-Constrained Environments}, pages = {62--73}, year = {2026}, editor = {Folorunso, Sakinat and Ogundokun, Roseline and Oladipo, Francisca}, volume = {319}, series = {Proceedings of Machine Learning Research}, month = {11--14 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v319/main/assets/adedokun26a/adedokun26a.pdf}, url = {https://proceedings.mlr.press/v319/adedokun26a.html}, abstract = {This study proposes a bimodal machine learning approach for multiclass deception detection by integrating facial Action Unit (AU) features and linguistic features extracted from a real-life dataset of video interviews of suspected criminal perpetrators, persons of interest, and convicted criminals. Facial features were obtained using the Facial Action Coding System (FACS), while linguistic cues were derived using Linguistic Inquiry Word Count (LIWC) scores. A mid-level data fusion strategy combines the extracted features into a unified representation. A Random Forest classifier applied to 3,720 real-life samples with 80:20 split and 10-fold cross-validation achieved an overall classification accuracy of 88%. Results confirm that combining facial and linguistic cues from real-life datasets provides a richer representation of deceptive behaviour.} }
Endnote
%0 Conference Paper %T Development of an Enhanced Machine Learning Model for Deception Detection Leveraging Facial Action Units and Linguistic Features %A Adetoye Oluwatoyin Adedokun %A Adebola K. Ojo %B Proceedings of IndabaX Nigeria 2026: Building Scalable AI That Works: From Research to Deployment in Resource-Constrained Environments %C Proceedings of Machine Learning Research %D 2026 %E Sakinat Folorunso %E Roseline Ogundokun %E Francisca Oladipo %F pmlr-v319-adedokun26a %I PMLR %P 62--73 %U https://proceedings.mlr.press/v319/adedokun26a.html %V 319 %X This study proposes a bimodal machine learning approach for multiclass deception detection by integrating facial Action Unit (AU) features and linguistic features extracted from a real-life dataset of video interviews of suspected criminal perpetrators, persons of interest, and convicted criminals. Facial features were obtained using the Facial Action Coding System (FACS), while linguistic cues were derived using Linguistic Inquiry Word Count (LIWC) scores. A mid-level data fusion strategy combines the extracted features into a unified representation. A Random Forest classifier applied to 3,720 real-life samples with 80:20 split and 10-fold cross-validation achieved an overall classification accuracy of 88%. Results confirm that combining facial and linguistic cues from real-life datasets provides a richer representation of deceptive behaviour.
APA
Adedokun, A.O. & Ojo, A.K.. (2026). Development of an Enhanced Machine Learning Model for Deception Detection Leveraging Facial Action Units and Linguistic Features. Proceedings of IndabaX Nigeria 2026: Building Scalable AI That Works: From Research to Deployment in Resource-Constrained Environments, in Proceedings of Machine Learning Research 319:62-73 Available from https://proceedings.mlr.press/v319/adedokun26a.html.

Related Material