Play Beyond Sound
Play Beyond Sound explores accessible communication systems for deaf and hard-of-hearing players in multiplayer games.
It introduces a new communication layer that translates urgency, tone, and intent into multi-sensory signals, making real-time interaction more perceivable, expressive, and inclusive.
In multiplayer games, voice chat carries more than typed words. Voice can communicate urgency, emotion, teamwork, and social presence in real time through auditory signals such as tone, timing, and volume. For deaf and hard-of-hearing players, many of these signals are not accessible, making it difficult to fully participate in fast-paced collaborative play.
Play Beyond Sound explores how voice-based social communication can be translated into visual and haptic forms. Drawing from accessibility research, gameplay analysis, and iterative prototyping, this thesis investigates how communication can become more visible, felt, and socially connected.
The project centers on an emotion-augmented caption system that visualizes tone, urgency, and speaker intent. Rather than only transcribing speech, it introduces expressive layers such as changes in size, weight, motion, and timing to help players interpret social cues. The system is supported by haptic feedback that translates voice and ping signals into vibration, and an expressive input system that enables Deaf and hard-of-hearing players to communicate back to teammates. A customization interface allows players to adjust how communication cues are displayed.
Play Beyond Sound reframes multiplayer communication as a multisensory and interactive system. It aims to support not only access to information, but also access to connection, coordination, and participation in team-based play.
