Skip to content
Publications

Article · Computers · October 23, 2026

Action-Unit-Constrained Graph Learning with Counterfactual Recourse for Auditable Facial Expression Recognition

Alpamis Kutlimuratov, Islambek Saymanov, Bekmurod Khujamuratov, Fazliddin Makhmudov, Turabay Djurayev, Navruza Irmukhamedova

Abstract

Facial expression recognition (FER) systems often explain predictions with saliency maps that neither identify specific facial behaviour nor indicate how a decision could change. We present AU-GRACE, which couples an action-unit (AU) relation graph with an explicit AU bottleneck to provide counterfactual recourse as small sets of plausible AU edits. Facial Action Coding System (FACS)/EMFACS knowledge supplies reliability-gated soft constraints, allowing departures from prior edges weakly supported by data. The emotion head accesses the image through the AU vector and a norm-penalised residual, so AU edits intervene in the decision. An amortised solver generates edits in 6.4 ms. AU-GRACE achieves 96.78%, 69.19%, and 94.92% accuracy on RAF-DB, AffectNet-7, and FERPlus, respectively, under the evaluation protocols defined by the dataset authors. Counterfactual validity is 94.8%, with 2.3 edited AUs and 86.4% plausibility. In a small expert study, agreement with six certified FACS coders reaches κ¯=0.71, close to their inter-coder agreement. The edits remain effective after rendering and re-classification by an independent model. These results show that, for the basic expression categories studied here, competitive recognition accuracy can be combined with AU-level recourse through an explicit AU bottleneck.

Emotion and Mood RecognitionExplainable Artificial Intelligence (XAI)Face recognition and analysisfacial expression recognitionfacial action unitsgraph neural networks

More by Islambek Saymanov