Harnessing medical twitter data for pathology AI

Summary: The lack of annotated publicly available medical images is a major barrier for computational research and education innovations. This study utilizes de-identified images and knowledge shared by clinicians on public forums to curate a large dataset called OpenPath, which consists of 208,414 pathology images paired with natural language descriptions. The researchers develop a multimodal artificial intelligence, PLIP, which is trained on OpenPath and achieves state-of-the-art performances for classifying pathology images. PLIP also enables users to retrieve similar cases by either image or natural language search, facilitating knowledge sharing.

Summary: This survey explores the development of vision-language pre-training (VLP) methods for multimodal intelligence in recent years. The approaches are categorized into image-text tasks, core computer vision tasks, and video-text tasks, with comprehensive reviews, case studies, and discussions on progress and challenges. Additionally, advanced topics actively explored in the research community are also discussed.