Research into Emotion Regulation
Within the AffecTech project, we aim to better understand and regulate emotions with the use of embodied sensors and personal technologies. As a team of researchers operating in the fields of Human-Computer Interaction (HCI), Digital Signal Processing, electrical engineering, neuroscience and clinical psychology, we’re driven to model common affective health disorders. In our research, we focus on anxiety, depression and bipolar disorders.
A critical area of our work considers methods for collecting and processing physiological data that can expose one’s affective patterns. A recent report, “Towards the Development of Sensor Platform for Processing Physiological Data from Wearable Sensors” from the GEIST.re group at the AGH University of Science and Technology examines the use of low-cost portable devices currently available for researchers in the field.
Biometrics and Affective Computing
Our work concerns the detection of emotional responses with the use of personal devices and physiological sensors; this closely relates to the aspects covered within the field of affective computing (AfC). The paper above cites previous research to support their use of Heart Rate (HR) and Skin Conductance/Galvanic Skin Response (GSR) signals when designing a computational model for human emotion recognition. These signals can be used to validate semantic emotional descriptors based on valence and arousal measurements, linked to the user’s involuntary reactions transmitted by the Autonomic Nervous System (ANS).
BITalino EDA/GSR sensor placed on the palm
Recent advancements in affordable wearable devices allow us to monitor such physiological signals with reasonable accuracy. This proposes a motivation to test how well they can imitate the quality of results acquired in lab experiments, supporting their potential in real-world applications.
What Technologies are Currently Available?
The paper compares the following low-cost portable devices:
These devices provide an accessible modality for data collection. They are affordable, mobile and provided with well-documented tools for research.
Requirements and Feature Evaluation
So, what makes a device fit for the purpose of affective health research?
- Accuracy. These devices are low-cost to be accessible to almost anyone. However, we do not want this to compromise the fidelity of our results.
- The device is expected to collect data continuously without interfering with the user’s day-to-day tasks. The platform must be mobile, comfortable for the user, robust in regards to prolonged sensor contact, and have a sufficient battery-life to last throughout the day.
- Connection to other devices e.g. through Bluetooth.
- Access to raw data that can then be filtered and processed post-recording without losing any data from the original signal.
It’s vital that the signals are recorded with a defined unit of measurement. This is so we can compare the results from different devices and accommodate the development of a general computational model. Ideally, the device should be provided with an open API that allows us to access the inputs.
The initial part of the test is used to evaluate the HR signals acquired from each device. During the experiment, the data from the Microsoft Band 2 reports the best average correlation with the reference signal, taken from the Polar H6 heart rate sensor. However, the authors notice that the HR measurements from the BITalino and the Empatica E4 are highly sensitive to external factors, such as device placement and the re-use of the ECG electrodes, causing a large spread in the correlation. In future experiments, appropriate precautions should be considered to relieve the vulnerability of these results.
Furthermore, during developments in the HR extraction algorithm, the authors are able to expose additional physiological information linked to the participant’s breathing patterns when using the BITalino and the Empatica E4. This data can be useful for further affective analysis, putting these two devices at a great advantage.
“While working on the HR extraction algorithm we noticed that using shorter analysis window and skipping final smoothing reveals HR variability (HRV). This is a great advantage of BITalino and Empatica E4 over the remaining devices. According to , during inhale the HR is growing and while exhale it is decreasing. This information may be used to extract breathing frequency and respiratory sinus arrhythmia (RSA). In future experiments both parameters may be used to examine the emotional state of the subjects.” Kutt et al.
The study also compares the GSR response from each of the 4 devices. The signals retrieved from the BITalino are highly similar to those by the eHealth sensors; they both produce signals of higher amplitude compared to the Empatica and MS Band. That said, the signals produced from the eHealth device are far noisier. It’s also noted that the technical support for this platform is currently being phased out, which is likely to hinder future developments, lending further preference towards the BITalino.
GSR signal comparison
In the final stage, participants are asked to engage in a physical activity (performing 20 squats) whilst their HR and GSR signals continue to be monitored. During these tests, the BITalino reports the lowest difference in correlation during stationary and physical activity. This is worth considering for applications that require additional sensor data such as Electromyography (EMG) or Accelerometer (ACC) which directly relate to the user’s physical behaviour. It’s also interesting from the perspective of AffecTech uses cases, which aims to continuously retrieve data “in the wild” i.e. when the user performs tasks as they would on a normal day-to-day basis.
And the Winner is…
The paper from AGH proposes a mobile platform for processing physiological data from wearable sensors. The authors express particular requirements as a means to benchmark a set of devices to be used in affective computing applications. A series of tests reveal which of these devices are most capable of producing high-quality results, and deemed suitable for detecting patterns in the participant’s emotional state within the context of real-world applications. After reflecting on the performance of each device, Kutt et al. favour the use of the BITalino platform (potentially in combination of the MS Band 2 to measure heart rate) in future research projects for emotion recognition frameworks. This conclusion is very exciting for the team here at PLUX, as we continue to promote and develop our platform for AffecTech’s research.
For more details regarding the study referenced in this post, please refer to the original paper:
Kutt, Krzysztof, Wojciech Binek, Piotr Misiak, Grzegorz J. Nalepa, and Szymon Bobek. “Towards the Development of Sensor Platform for Processing Physiological Data from Wearable Sensors.” In International Conference on Artificial Intelligence and Soft Computing, pp. 168–178. Springer, Cham, 2018.
The paper was produced by the GEIST.re research group
Bandreader software: https://geist.re/pub:software:bandreader
AffCAI research group: https://affcai.eu/
This article was created by Wiliam Primett, a member of the AffecTech research team who works at Plux Wireless Biosignals. William is an AffecTech Early Stage Researcher (ESR) and PhD student who’s work involves designing tools for exploring non-verbal expression and collaborative interaction design.
AffecTech is a digital health research network advancing personal health technologies to help people with affective health disorders (depression, anxiety and bipolar disorder). AffecTech is a project funded by the European Union under the Horizon 2020 Marie Skłodowska-Curie Innovative Training Networks. www.affectech.org
AffecTech has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 722022.
Note: All images in this article are used with kind permission.