Human communication is a deeply expressive tapestry relying on far more than spoken words alone. Everyday interactions are continuously shaped by subtle physical cues, such as nodding in agreement, shrugging shoulders to indicate uncertainty, or waving hello to welcome a friend. For individuals living with severe paralysis, restoring this multifaceted level of interaction has long remained an elusive clinical goal. Brain-computer interface technology has advanced significantly in recent years, enabling patients to type messages on screens, control robotic cursors, or generate synthetic speech. However, nearly all of these groundbreaking systems have operated on a singular track, forcing users to choose between verbal output and physical movement rather than supporting both simultaneously. A new technological development led by researchers at the University of California, San Francisco, has addressed this limitation, introducing an experimental neural prosthesis capable of translating both verbal speech and non-verbal body gestures simultaneously and within seconds of user intent.
Implant design captures complex neural signals
The foundation of this dual-mode communication breakthrough relies on an extensive surgical approach designed to capture overlapping electrical activity across a broad area of the brain. Described in Nature Neuroscience, the experimental hardware utilises an array of 253 electrodes placed on the surface of the sensorimotor cortex. This region of the brain plays a critical role in orchestrating both physical movement and vocalisation. Because human communication naturally blends these channels, decoding them concurrently presents a formidable challenge, as the underlying neural signals frequently overlap when performed together compared to isolated tasks. To examine this, the research team worked with two clinical trial participants who had lost the ability to speak and move their limbs due to severe paralysis. The first participant, designated Bravo-1r, experienced paralysis following a brainstem stroke, while the second participant, Bravo-6, lives with amyotrophic lateral sclerosis. By recording brain activity while these individuals repeatedly attempted to speak specific phrases and perform distinct physical gestures either independently or concurrently, the investigators gathered data needed to build responsive decoding systems.
Machine learning powers digital avatar output
Once the neural data was captured, the scientific team deployed customised machine learning models to translate patterns of brain activity into visible actions. These algorithms were trained to interpret the unique neural signatures of each participant, converting their intended thoughts into text displayed on a screen while simultaneously driving a personalised digital avatar. The resulting performance metrics demonstrated proof of concept across both trial participants. For Bravo-1r, the technology predicted intended gestures and spoken phrases with accuracy rates of 88 percent and 84 percent, respectively. For Bravo-6, the system attained gesture accuracy of 66 percent and speech accuracy of 70 percent. Independent computational neuroscientists have welcomed these findings as an encouraging milestone that brings medical science closer to creating neural prostheses suitable for daily human interaction, though experts note that vocabulary sets remain modest and error rates require further reduction.
