University of Birmingham's JAMS platform brings live music collaboration and education to VR
The University of Birmingham has announced the development of the Joint Active Music Sessions (JAMS) platform, a virtual solution designed to replicate the experience of playing music in person.
JAMS uses personalized avatars to enable musicians to collaborate in real-time for performances, practice sessions, or music teaching.
Dr. Massimiliano Di Luca from the University of Birmingham explained:
“A musician records themselves and sends the video to another musician. The software creates a responsive avatar that plays in perfect synchrony with the music partner. All you need is an iPhone and a VR headset to bring musicians together for performance, practice, or teaching.”
The platform offers immersive features such as real-time adaptability and personalized experiences. By capturing subtle performance cues like a violinist's bow movements or maintaining eye contact, JAMS aims to replicate the nuances of live collaboration.
A key feature of JAMS is its elimination of latency—a common issue in digital music platforms. According to Dr. Di Luca:
“Latency is the delay between a sound production and when it reaches the listener, and performers can start to feel the effects of latency as low as 10 milliseconds, throwing them ‘off-beat,’ breaking their concentration, or distracting them from the technical aspects of playing.”
The technology is supported by the Augmented Reality Music Ensemble (ARME) project, a multidisciplinary initiative combining expertise from psychology, computer science, engineering, music, sport science, and mathematics. The ARME project's algorithm enables dynamic timing adjustments, ensuring seamless synchronization between avatars.
JAMS has wide-ranging applications, from interactive music groups and teaching to commercial uses such as lipsyncing and dubbing. Additionally, the platform can collect user data to create digital twins of musicians, opening licensing opportunities for catalog and publishing rights.
Dr. Di Luca summarized the vision behind JAMS:
“We’re aiming to bring the magic of playing music in person to the virtual world. You can adapt the avatar that other people play with, or learn to play better through practice with a maestro.”
