Sony Patent Turns Game Audio Into Visual Cues for Deaf and Hard-of-Hearing

A newly published Sony Interactive Entertainment patent application, titled "Audio Processing System and Method," describes a way to turn in-game sounds into visual cues for players who are deaf or hard of hearing. The system classifies the intended effect of a sound before it plays, such as building fear or tension, warning of danger, or describing the environment, and then displays a matching visual element. That could be an icon, a shape, or even an onomatopoeic word like "Roar!" for a nearby monster. Properties such as color, brightness, or transparency are tied to the sound's volume and character.

The patent also describes directional lighting to show where a sound is coming from, either as LEDs around the display or as LEDs encircling the touchpad on a DualSense controller. The application, published on October 22, is a continuation of a US application filed in May 2023, which in turn claims priority from a UK filing dating back to June 2022. One interesting detail is support for older games, since the patent suggests the operating system or a helper app could tag a legacy game's sound files and generate the cues without changes to the game's code. Read full story