Over the past decade, artificial intelligence (AI) for science has catalyzed diverse interdisciplinary research and driven significant advancements in fields such as life science, astronomy, and neuroscience. Particularly, the integration of AI has greatly revolutionized scientific and clinical practices, including disease diagnosis, robotic surgery, and drug discovery, by leveraging diverse data modalities like medical images and videos, thus enhancing overall healthcare. Despite AI's undeniable potential in medicine, its implementation faces several key barriers, including the need for large datasets, extensive labeling, and the labor-intensive design of specialized systems. Overcoming these challenges is crucial for the effective development and deployment of AI in healthcare.

We have proposed innovative methods and frameworks to address these challenges in AI for medicine, including (1) secure data sharing approaches among multiple medical centers to achieve large-scale data collection without privacy concerns, (2) robust deep learning algorithms that require only weak annotations to reduce the labeling workload, and (3) a single, multi-modal, multi-task foundation model capable of adapting to various data modalities and medical tasks, thereby avoiding repeated AI development. These foundational advancements are poised to make AI a more versatile and accessible tool in the field of medicine, supporting both research endeavors and clinical applications.

Representative publications

• Multi-grained Radiology Report Generation with Sentence-level Image-language Contrastive Learning, IEEE Transactions on Medical Imaging 2024

• Robust deep learning from incomplete annotation for accurate lung nodule detection, Computers in Biology and Medicine 2024

• Disorder-Free Data Are All You Need—Inverse Supervised Learning for Broad-Spectrum Head Disorder Detection, New England Journal of Medicine AI 2024

• Pan-mediastinal neoplasm diagnosis via nationwide federated learning: a multicentre cohort study, The Lacent Digital Health 2023

• Automatic intracranial abnormality detection and localization in head CT scans by learning from free-text reports, Cell Reports Medicine 2023

• Relay Learning: A Physically Secure Framework for Clinical Multi-site Deep Learning, NPJ Digital Medicine 2023

• Multi-modal Contrastive-Generative Pre-training for Fine-grained Skin Disease Diagnosis, IEEE International Conference on Bioinformatics and Biomedicine 2023

• RATING: Medical knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning, Patterns 2022

• A Digital Mask to Safeguard Patient Privacy, Nature Medicine 2022

• Deep Learning with Weak Annotation from Diagnosis reports for detection of multiple head disorders: a prospective, multicentre study, The Lancet Digital Health 2022

• Toward human intervention-free clinical diagnosis of intracranial aneurysm via deep neural network, Patterns 2021