Tang, J., LeBel, A., Jain, S., & Huth, A. G. (2022). Semantic reconstruction of continuous language from non-invasive brain recordings (p. 2022.09.29.509744). bioRxiv

fMRI-based decoder of continuous natural language, tested on perceived speech, imagined speech, and silent videos.

Involved 16hrs of per-subject training data.

Uses a semantic language decoder to compensate for the low temporal resolution of fMRI, so ordinary word error rate metrics (0.92-0.94) are misleading: decoded sentences are more about capture the “gist” of what’s intended.