Brain-computer interface

Collecting feats:

• August, 2023: ECoG (!) speech decoding (via motor—attempted speech) with 25% WER @ 78 WPM on a 1024-word vocabulary set. Also attempts synthesized speech (modest results) and avatar facial expression synthesis (usable results)
  • Metzger, S. L., Littlejohn, K. T., Silva, A. B., Moses, D. A., Seaton, M. P., Wang, R., Dougherty, M. E., Liu, J. R., Wu, P., Berger, M. A., Zhuravleva, I., Tu-Chan, A., Ganguly, K., Anumanchipalli, G. K., & Chang, E. F. (2023). A high-performance neuroprosthesis for speech decoding and avatar control. Nature, 620(7976), 1037–1046. https://doi.org/10.1038/s41586-023-06443-4
• 2023-01-24: Intracortical BCI using microelectrode array, attempted and internal speech. Achieved 9.1% WER on a 50 word vocabulary and 23.8% WER on 125k word vocabulary (“first successful demonstration of large-vocabulary decoding”). Decoded speech at 62 WPM. All these figures are integer-factor improvements over previous SotA.
  • Willett, F. R., Kunz, E. M., Fan, C., Avansino, D. T., Wilson, G. H., Choi, E. Y., Kamdar, F., Glasser, M. F., Hochberg, L. R., Druckmann, S., Shenoy, K. V., & Henderson, J. M. (2023). A high-performance speech neuroprosthesis. Nature, 620(7976), 1031–1036
• November, 2022: intracortical BCI, internal speech (rather than mimed/attempted speech) distinguishing eight words at 91% accuracy with single neuron supramarginal gyrus electrode
  • Wandelt, S. K., Bjånes, D. A., Pejsa, K., Lee, B., Liu, C., & Andersen, R. A. (2022). Online internal speech decoding from single neurons in a human participant (p. 2022.11.02.22281775). medRxiv. https://doi.org/10.1101/2022.11.02.22281775
• November, 2022: intracortical BCI, attempted speech in paralyzed patient, typing at 29.4 characters / minute
  • Metzger, S. L., Liu, J. R., Moses, D. A., Dougherty, M. E., Seaton, M. P., Littlejohn, K. T., Chartier, J., Anumanchipalli, G. K., Tu-Chan, A., Ganguly, K., & Chang, E. F. (2022). Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis. Nature Communications, 13(1), 6510. https://doi.org/10.1038/s41467-022-33611-3
  • “We designed a sentence-spelling pipeline that enabled a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with anarthria and paralysis to silently spell out messages using signals acquired from a high-density electrocorticography (ECoG) array implanted over his sensorimotor cortex (Fig. 1)”
• September, 2022: fMRI-based Silent speech interface, internal (“imagined”) speech and perceived speech; semantic decoding
  • Tang, J., LeBel, A., Jain, S., & Huth, A. G. (2022). Semantic reconstruction of continuous language from non-invasive brain recordings (p. 2022.09.29.509744). bioRxiv
• 2021-05-12: intracortical BCI, imagined handwriting, typing at 90 characters / minute
  • High-performance brain-to-text communication via handwriting. Francis R. Willett, Donald T. Avansino, Leigh R. Hochberg, Jaimie M. Henderson & Krishna V. Shenoy
  • n.b. typical smartphone typing speed for participants’ age group is 115 characters / minute, but for 10-29 y.o. U.S. subjects is ~195, per Palin, K., Feit, A. M., Kim, S., Kristensson, P. O., & Oulasvirta, A. (2019). How do People Type on Mobile Devices?: Observations from a Study with 37,000 Volunteers. Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, 1–12. https://doi.org/10.1145/3338286.334012