Kim, T. S., Latzke, M., Bragg, J., Zhang, A. X., & Chang, J. C. (2023). Papeos: Augmenting Research Papers with Talk Videos. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 1–19. UIST ’23: The 36th Annual ACM Symposium on User Interface Software and Technology

An Augmented reading interaction design aimed at linking article and video representations of research artifacts. Design elements:

• Segments of the video are associated with blocks of the article.
• Thumbnails of those segments are presented in the gutter alongside those regions.
• A small affordance near each linked block in the paper allows rapid scrubbing through the associated video clip.
• Key sentences can be bidirectionally linked between the article and the video transcript, so that clicking one navigates the other representation accordingly.
• A “top-level” video player can be used to “play” the entire paper, scrolling to successively linked passages in the article as the video plays.
• An authoring interface is included.
  

The design is implemented in the Semantic Reader via PaperCraft

Papeo Project @ CSCW 2022
Demos