In this paper, we investigate recognizing context over time using physiological signals. Using the CASE dataset we evaluate both unimodal and multimodal approaches to physiological-based context recognition, over time. For recognition, we evaluate a random forest, as well as state-of-the-art neural network. These classifiers are evaluated using accuracy, Kappa, and F1-Macro metrics. Our results suggest that the fusion of EMG signals is more accurate, at recognizing context over time, compared to the fusion of non-EMG physiological signals. Although the fusion of non-EMG has a comparatively higher accuracy, ECG data results in the highest unimodal accuracy. Considering this, we analyze how the signals are correlated, including when the are fused (i.e. multimodal). We also perform a cross-gender analysis (e.g. training on male data and testing on female data) suggesting some generalizability across gender.

In this paper, we propose a method for pain recognition by fusing physiological signals (heart rate, respiration,blood pressure, and electrodermal activity) and facial actionunits. We provide experimental validation that the fusion ofthese signals results in a positive impact to the accuracy ofpain recognition, compared to using only one modality (i.e.physiological or action units). These experiments are conductedon subjects from the BP4D+ multimodal emotion corpus, andinclude same- and cross-gender experiments. We also investigatethe correlation between the two modalities to gain furtherinsight into applications of pain recognition. Results suggestthe need for larger and more varied datasets that includephysiological signals and action units that have been codedfor all facial frames.