Movie music magic

An EPSRC-supported doctoral student has developed pioneering technology to enhance mood music in film. The research is focused on enriching the musical experience of film audiences and might also help the hard of hearing.

  • Mood Glove, a haptic wearable prototype, harnesses the wearer’s sense of touch to provide them with information; this adds another layer of expression to the film experience
  • This system suggests simple emotions to intensify moods in film music that tie in with action
  • Mood Glove was showcased at high profile international events and featured in national UK press

For the hearing impaired, watching a movie like the 1993 classic The Piano can be very frustrating. Subtitles make it easy enough to follow the dialogue and plot, but the inability to experience the music – often so important to a film – means much of the emotional content is inevitably lost. Antonella Mazzoni, a student at the EPSRC/AHRC Centre for Doctoral Training (CDT) in Media and Arts Technology, believes that her approach could have positive effects on the hard of hearing, too.

Her research looked at how music is used in movies to enhance both mood and emotions, and how enriching the musical experience of audiences through haptic sensations might also have implications for hearing impaired people, providing them with a new, enhanced emotional experience while watching a film.

Antonella’s aim was to find a solution that was not only affordable but also allowed users a degree of freedom of movement and avoided the need to confine themselves to a particular chair. The result was a ground-breaking piece of wearable digital technology – a glove inducing simple emotions, such as calmness, happiness, sadness and alarm – that tie in with the action unfolding on the screen.

The glove, named Mood Glove, utilises so-called ‘haptic’ feedback. This means the sense of touch is harnessed to provide information to the wearer.

Antonella argues that the design of haptic sensations for media enhancement should be an artistic process, rather than an effect linked to automatically extracted features. She envisions haptic compositions to be created for a movie just as a composer scores the music for it. In her last study Antonella designed haptic sensations and used Mood Glove to target the build-up of tension on screen. As well as providing a much more immersive experience for anyone watching a movie, it can add a layer of experience previously inaccessible to people with hearing impairments.

Antonella Mazzoni has showcased Mood Glove at the 2015 International Culture Makers’ Joint Design Exhibition ‘Utopia of Culture Makers’ in Milan; the 2015 London Wearable Tech Show; Digital Shoreditch 2015; the Wearable Technology Festival at the Museum of Science and Industry in Manchester 2016; the 2016 ACM Conference on Designing Interactive Systems; and Inter/Sections 2016. Her work was subsequently covered in the Daily Mail and exhibited at the National Science and Media Museum in Bradford as part of the ‘Supersenses’ exhibition in 2017.