Wellington disabled sound artist Jesse Austin-Stewart has developed Music for PlayStation, a collection of five musical vibration pieces designed to be played through the PlayStation DualSense controller.

Music for PlayStation resulted from Jesse’s PhD in spatial audio. Spatial audio is where composers move sound around the listener’s head.

The difficulty with spatialisation – moving sound around someone’s head – is that often, you need to be a hearing individual to hear what the composer intended, Jesse explains.  

“Going through Music School, I had a number of friends in the hard of hearing and d/Deaf communities who were directly affected by this. So as a part of my PhD research, I worked with them to understand the barriers they felt when engaging with musical activities and spatial audio, and asked what kind of solutions they thought would work.

Predecessor to Music for PlayStation

This resulted in a predecessor to Music for PlayStation called Spatial Vibrations, a custom piece of hardware that musically vibrated along different points of the user’s arm.

“I showed this to members of the hard of hearing and d/Deaf communities I was working with, and they felt it removed all the barriers they had previously found and felt as if they could engage in a similar fashion to hearing individuals.

“This was really exciting, as it felt like a step towards removing barriers that inhibit people from engaging with spatial audio.”

The problem, which led to Music for PlayStation, was that the interface (programme) Jesse had built was custom-made and therefore expensive; required other technology to run; and required the user to have the knowledge to set it up.

Engaging with hard of hearing and d/Deaf communities 

So Jesse decided to use the haptically advanced (vibrating) PlayStation DualSense controller. It is cheaper than a custom interface and available to the general public.

To compose Music for PlayStation, Jesse engaged with the same members of the hard of hearing and d/Deaf communities with whom he had worked on Spatial Vibrations.

“The result is that the five tracks that make up the EP can be played back on a vibration interface – the controller – that’s relatively cheap and easy to set up, promoting a less exclusive spatial audio for audiences of all types of hearing.

“As a disabled person myself, I am constantly aware of how barriers exclude people, and my PhD sought to promote inclusion,” he says. “This music has been designed so that audiences of all types of hearing can engage with the music on equal terms.

“After plugging in your controller and setting it up, all you need to do is press play on the audio on your computer, and you will start to feel the controller vibrate in your hands. Feel the way that the rhythms move from left to right and the way the changing patterns feel against your palms.”

Music for PlayStation will be released on 25 November 2022, and will be available to stream on all popular streaming sites as well as free download on Bandcamp. Jesse’s website has set-up instructions for both Windows and Mac computers.