AES Europe 2026

The ability to objectively measure listener emotion is a
critical frontier for adaptive audio systems, healthcare,
; personalized music therapy. While music is a powerful
driver of affect, traditional self-reporting is often
intrusive or inaccessible for users in wellbeing settings
who may struggle to articulate their mood. This paper
introduces JoyCam, a multimodal system that estimates
subtle moments of joyful engagement by blending lightweight
brain-wave monitoring (wearable EEG) with facial-expression
sensing. By capturing physiological reactions that occur
below the threshold of conscious awareness, the system
creates a more stable emotional profile than
single-modality methods. In our system, Facial joy is
estimated via MediaPipe landmark analysis, focusing on
normalized mouth-width deviations. Simultaneously,
neurological engagement is tracked through Frontal Alpha
Asymmetry (FAA) using an OpenBCI Cyton system. To address
the sensitivity of EEG to movement, a dynamic artefact
index down-weights neural signals during high-frequency
interference. The system was tested in a pilot study with
five participants. Preliminary results indicate that
baseline-corrected physiological scores align closely with
self-reported music impact; valence ratings across
joyful; sad conditions. These findings suggest that
JoyCam offers a robust framework for responsive musical
companions that can adjust playlists or production
parameters based on a listener’s real-time physiological
state