Combining eye tracking and physiology for detection of emotion and workload

Peripheral physiological measures such as electrodermal activity (EDA), heart rate and pupil dilation, as well as neurophysiological measures such as electroencephalography (EEG), can inform us about individuals’ cognitive and emotional state. We are interested in exploiting such measures in real life situations. A challenge of interpreting physiological measures as markers of mental state in real life is the lack of context information. We here approach this challenge by relating physiological measures to eye tracking. Participants scanned stimuli that induced different levels of workload (small sets of numbers that needed to be added or not) and different types of emotion (neutral, pleasant and unpleasant pictures). EDA, heart rate, pupil size and EEG were related to the first eye fixation on the stimulus. Our first results indicate that this may be especially helpful in situations related to cognitive workload, e.g., determining whether operators are not only looking at, but are also cognitively processing information that is presented on a screen.

Measuring Behavior 2020-21: 12th International Conference on Methods and Techniques in Behavioral Research and 6th Seminar on Behavioral Methods