Real-time feedback of subjective affect and working memory load based on neurophysiological activity

We investigated the effects of feedback on users’ performance during a cognitive task with concurrent emotional distraction. Our aim was to provide participants with insights into their current affective and cognitive state by measuring and decoding brain activity. Therefore, a real-time preprocessing, analyzing, and visualization routine was developed based on electroencephalographic (EEG) data measured during a primary study. To explore users’ behavioral and neurophysiological reactions, error-tolerance as well as possibilities to improve feedback accuracy by the means of feedback-based event-related potentials (ERPs), we provided either legit or inappropriate sham feedback in a second study. The kind of feedback (legit or inappropriate) had only marginal influence on participants’ subsequent performance. On a neuronal level, we did not observe differences in the ERPs evoked by the legit and inappropriate feedback. In qualitative interviews, participants evaluated the feedback as interesting but also sometimes irritating due to odd feedback trials. Our study emphasizes the importance of performance accuracy and transparency towards users regarding the underlying feedback computations.

HCI International 2021