For years, Pancho communicated by spelling words on a computer with a pointer attached to a baseball cap, a tedious method that allowed him to type out about five correct words per minute.
“I had to bend / lean my head forward and put in one key letter at a time to write it,” he wrote via email.
Last year, the researchers gave him another device with a head-controlled mouse, but it’s still not nearly as fast as the brain electrodes in the research sessions.
Pancho communicated 15 to 18 words per minute through the electrodes. That was the maximum rate the study allowed because the computer waited between prompts. Dr. Chang says faster decoding is possible, although it’s unclear whether it will reach the pace of a typical conversation: around 150 words per minute. Speed is a major reason the project focuses on speaking by typing directly into the brain’s word production system, rather than hand movements while typing or writing.
“It’s the most natural way to communicate,” he said.
Pancho’s lively personality helped researchers overcome challenges, but it also occasionally makes speech recognition erratic.
“Sometimes I can’t control my emotions, I laugh a lot and don’t do the experiment so well,” he wrote by email.
Dr. Chang recalled times when, after the algorithm successfully identified a sentence, “he was visibly shaking and it looked like he was giggling.” When that happened, or when he yawned or became distracted during repetitive tasks, “did.” it doesn’t work very well because he wasn’t really focused on understanding those words. So we have to work on some things because of course we want it to work all the time. “
The algorithm sometimes confused words with similar phonetic sounds and identified “go” as “bring”, “do” as “you” and words beginning with “F” – “believe”, “family”, “feel” – as a V word “very”.