The highs and lows of this years Australian Open have been delighting fans around the world, especially during global lockdowns. But not everyone is able to enjoy the thrill of live sports in the same way. For the tennis fans among the 600,000 blind and visually impaired people in Australiaand millions more around the worldexperiencing sports broadcasts can be underwhelming. Kala Petronijevic, a young girl who has low vision, says her dad has to commentate every move to her. Maurice Gleeson, a blind tennis fan and the CEO of Blind Sports and Recreation Victoria, says: Tennis, sadly, hasnt been anywhere near as accessible as I would like.

But, this weekend, the blind and low vision community (BLV) will have a brand-new way to experience the upcoming mens and womens finals.

[Image: courtesy AKQA]For many months, digital agency AKQA and researchers from Monash University in Melbourne, have been finding a way to make watching tennis more accessible. With investment from Tennis Australia, the countrys tennis governing body, theyve developed Action Audio, which takes visual, spatial data from the tennis court and converts it into audio.

Its a confluence of two existing technologies, says Tim Devine, AKQAs executive creative director. The spatial data is captured by Hawk-Eye, a computer vision system already used in various sports, which is made up of a series of high-speed cameras that track the trajectory of a ball in real time. Its often referred to as electronic line calling (or goal-line technology in soccer). In fact, for the first time the Australian Open has this year done away with human line judges in favor of these robo-referees.

The new system converts that spatial data into 3D audio, or binaural sound. Thats the best way to digitally hear audio as we do in real life, where we instinctively know where, exactly, a sound is coming from. Its a much more complex version of the older concept of surround sound. The technology has lately been developing fast due to its usefulness for virtual reality.

See what its like during a tennis match:

In tennis, that means people can hear different audio sounds according to exactly what the ball is doing, instead of just hearing an indistinguishable mono sound whenever the ball hits the ground. The team wanted to re-create the sensation of hearing a ball flying into the corner of the court at 180 kilometers (more than 110 miles) per hour, for example. Its incredible, and I think you miss out on that if you just hear a hit, Devine says.

Now try it without any visuals:

With Action Audio, different elements of the game have different sound cues. (They used some of the concepts from blind tennis, Devine says, where the ball often contains beads to make it rattle or jingle.) You hear a blip each time the ball hits the ground, in your left or right ear, depending on where the ball is. The closer the ball is to the perimeter of the court, the more blips you hear. As the ball moves around in the air, you hear a jingling bell sound. You hear a high pitch for forehands and a low pitch for backhands.

I can actually hear everythingand I can see the ball, for some reason, says Petronijevic, after experiencing the new system. I wouldnt watch it any other way, says Adam Fayad, who also has low vision. You can just enjoy the game so much better.

[Photo: courtesy AKQA]Action Audio was already available for the wheelchair singles final, and will be available for both the mens and womens singles finals this weekend. People can tune in digitally on the Tennis Australia website; Australian residents can also hear it on Vision Australia Radio, a collection of stations that offer programming for the visually impaired. All thats needed is a regular set of headphones (it will work without them, but the experience is less immersive).

For the developers, tennis seemed like a good place to start for sonifying sport, since it involves only two players, with few other distractions. Itll be more complex for sports with more elements, like soccer or cricket, since the technology is fundamentally about deconstructing the experience to all the individual actions that contribute to a live sport experience and finding a way to represent them sonically.

The key may be in a new app that the team has developed. Using similar audio recognition techniques as Shazam, the app works to detect any live sports broadcast and overlay the 3D audio technology onto that game. It would essentially create a secondary audio stream for the broadcast. This isnt a difficult thing for broadcasters to pick up, Devine says, adding that its most important to keep working with the BLV community to hear what they need to create the most meaningful experiences. Now, its time to continue to listen again, he says, and get people to help shape the next sport, the next iteration.

See the original post here:

New technology allows blind people to experience live tennis - Fast Company