Cells are the fundamental units of life. Many of their functions and intercellular interactions are only manifested in different physiological and pathological states within living animals, making them difficult to replicate in vitro. Therefore, achieving long-term, high-speed, high-resolution, three-dimensional imaging of large-scale cell populations in situ in living animals will provide an unprecedented panoramic view for understanding complex life processes such as neuroscience, immunology, and drug responses, leading to new discoveries.

National academician Qionghai Dai's team at Tsinghua University has long been dedicated to research in the field of mesoscopic in vivo bioimaging. With the support of a major instrument project from the National Natural Science Foundation of China in 2013, the team successfully developed the world's first gigapixel-level wide-field mesoscopic fluorescence microscope (Real-time, Ultra-large-Scale, High-resolution, RUSH¹), which has been hailed as a pioneer in the field of mesoscopic microscopy. Over the past seven years, the team further innovated in computational imaging principles and achieved ultra-fine four-dimensional perception and reconstruction of incoherent light fields through scanned light field imaging², and proposed a digital adaptive optics architecture. By recording high-dimensional imaging processes and employing the adaptive-optics reconstruction, they broke through the inherent limitations of traditional imaging designed for the human eye's "what you see is what you get" paradigm. They have successively overcome barriers in in vivo imaging such as optical aberrations³, three-dimensional motion artifacts⁴, background fluorescence⁵, photon noise^6-8, and tissue scattering⁹. Integrating these advancements into a single architecture, they developed the new-generation mesoscopic in vivo microscope, RUSH3D¹⁰. This has been lauded as a leap forward in mesoscopic imaging and was selected as one of the "Top Ten Science and Technology Progress News in China in 2024 as judged by academicians from both academies." However, the rapid improvement in imaging performance has also imposed higher demands on large-scale data processing. Supervised deep neural networks can accelerate the three-dimensional reconstruction process but face challenges with generalization, making them difficult to apply broadly to various samples in complex environments. How to achieve both "trustworthy" and "speed," enabling instrument users to obtain high-fidelity, high-resolution three-dimensional information in real-time without needing to retrain new neural networks, remains challengeable hindering the widespread adoption of computational imaging microscopy.

On May 12, 2025, Qionghai Dai and Jiamin Wu from Tsinghua University, along with Jingyu Yang from Tianjin University, as co-corresponding authors, published an article in Nature Methods entitled "Physics-driven self-supervised learning for fast high-resolution robust 3D reconstruction of light-field microscopy." Addressing this unresolved challenge, the research developed a physics-driven self-supervised 3D reconstruction network, SeReNet. This network can achieve high-fidelity, high-resolution light-field 3D reconstruction at millisecond-level real-time processing speeds, without the need for paired training data. SeReNet fully integrates physical imaging priors into the network training by progressively reducing the distance between the network's predicted reprojection and the original acquisition. This guides the network to perform reconstruction inference based on the original acquisition, rather than over-generating details based on end-to-end data mapping. The research team specifically considered common issues in actual in vivo observation scenarios, such as strong noise, non-rigid sample motion, and optical aberrations. This allows SeReNet, using only a pre-trained network, to be widely applied to all common samples, effectively combating interference from complex imaging environments like aberrations and noise. The team validated SeReNet's high-speed, high-fidelity 3D reconstruction capabilities through simulation tests and experiments on various model animals, including zebrafish embryo development, zebrafish larval immune responses, Dictyostelium discoideum population interactions, Caenorhabditis elegans neural recording, and mouse liver injury. For the first time internationally, they achieved ultra-long-term, subcellular-resolution, high-speed, three-dimensional, multi-color imaging and real-time 3D reconstruction continuously for several days.

Figure1：Principle of SeReNet

Light-field microscopy achieves simultaneous acquisition of multi-view images through optical design. This can be simply understood as a specific superposition of multi-angle information and defocus blur. Its reconstruction process involves inputting the captured light field and outputting three-dimensional volumetric representation of the observed scene. Inspired by "refocusing", SeReNet first back-projects multi-angle features (depth decomposition module) into 3D space according to their respective geometric view directions to align them. Then, a 3D convolutional neural network (deblurring and fusion module) performs multi-view fusion to produce the final 3D reconstruction. During training, SeReNet uses the physical optical point spread function (PSF) to reproject the network's prediction back into the 4D light-field space domain. Parameter optimization are achieved by minimizing the difference between the reprojected light field and the input light field (self-supervised module). The core design philosophy of SeReNet is to leverage the lightweight and high information density characteristics of multi-view geometric optics priors to design the network's forward inference, ensuring “speed”. Simultaneously, it utilizes the accuracy of PSF physical priors to design the backward training process to enhance precision. This ultimately achieves high-speed, high-resolution, self-supervised 3D reconstruction (approximately 20 frames per second for 3D volume size of 429×429×101).

Reconstruction of SeReNet in various live biological experiments

Figure 2：SeReNet reconstruction in zebrafish embryo development, zebrafish larval immune responses, Dictyostelium discoideum population interactions, Caenorhabditis elegans neural recording

Migrasomes are organelles discovered in the 21st century. They are very small, only at the sub-micrometer scale, and require high-resolution observation. SeReNet successfully recorded the formation process of two migrasomes during zebrafish embryo development. Furthermore, imaging experiments such as the information exchange behavior of amoeba swarms via extracellular vesicles and the long-term spontaneous neural activity of intact C. elegans further validated that SeReNet can still achieve high-speed, high-definition three-dimensional observation in actual in vivo scenarios.

Recording of in vivo subcellular activity in mice after liver injury

Figure 3：Recording of interactions between immune cells and endothelial cells in mouse liver injury experiments

The research team used SeReNet to observe the mechanisms by which immune cells repair damage in vivo after mouse liver injury. In Liver Ischemia-Reperfusion Injury (LIRI) procedures, compared to the control group, the number of neutrophils in the experimental group significantly increased. Notably, SeReNet observed an interactive behavior where a neutrophil was pulled by the contractile fibers of a macrophage. Interestingly, migrasomes produced at the tail end of the neutrophil also generate contractile fibers that transport them into the macrophage. Another type of liver damage is Acetaminophen-Induced Liver Failure (AILF). Sixteen hours after the injection of acetaminophen, SeReNet observed endothelial cells recruiting monocytes and their mutual adhesion. Considering that recruitment and adhesion are common modes of intercellular communication, it was concluded that signal transduction might exist between endothelial cells and immune cells, potentially being one of the factors that exacerbate liver injury in mammals. Therefore, targeting CD63+ endothelial cells is expected to provide new insights for developing effective therapies for drug-induced liver failure.

2-day-long continuous 3D observation of immune inflammatory storm

Figure 4：Recording and analysis of whole-body immune cell activity in zebrafish larvae following tail transection surgery

Figure 5：Cells tracking of whole-body immune cell activity in zebrafish larvae: tail transection surgery and control groups

In living organisms, immune inflammatory storms typically last for several days. During this process, immune cells are very active, necessitating dynamic monitoring with millisecond-level precision. This time scale, spanning 4-5 orders of magnitude, results in a data volume of hundreds of thousands of frames, posing enormous challenges for imaging and reconstruction. SeReNet precisely fills this gap. The research team conducted continuous imaging of the immune response in zebrafish larvae for a total of 48 hours after tail injury, generating over 345,600 timepoints awaiting reconstruction. Traditional methods would require about two years to see the final results, whereas SeReNet can reconstruct all the three-dimensional volumetric data within a week, making large-scale three-dimensional observation and analysis of biological experiments possible.

Core patents based on this series of achievements have been transferred from Tsinghua University to establish a domestically developed advanced microscopy instrument company, Zhejiang Hehu Technology Co., Ltd. The company is dedicated to producing domestically developed and controllable high-end optical microscopes with internationally leading performance and advancing their cutting-edge applications in life sciences and other fields. It has already supported leading domestic research institutions such as Tsinghua University, Peking University, Beihang University, Beijing Normal University, PLA General Hospital, and Tongji Hospital in conducting over 20 innovative life science research projects in diverse fields including oncology, immunology, and neuroscience. These efforts serve areas such as life science discovery, basic medicine, and biopharmaceuticals.

Lu, Zhi, a postdoctoral fellow in the Department of Automation at Tsinghua University, and Jin, Manchang, a joint PhD student from the Future Information Innovation Institute of Fudan University and Shanghai Innovation Institute, are the co-first authors of this article. Academician Dai, Qionghai and Associate Professor Wu, Jiamin from the Department of Automation at Tsinghua University, and Professor Yang, Jingyu from the Department of Communication at Tianjin University are the co corresponding authors of this article. Chen Shuai, Wang Xiaoge, Sun Feihao, Zhang Qi, and Zhao Zhifeng participated and made important contributions. This work has received strong support from the National Natural Science Foundation of China, the Beijing Natural Science Foundation, the Key R&D Program of the Ministry of Science and Technology, the Tsinghua Fuzhou Joint Data Technology Research Center, the National Postdoctoral Innovation Talent Support Program, the China Postdoctoral Science Foundation, the Tsinghua Shuimu Scholars Program, and the Qinghua University Peking University Joint Center for Life Sciences.

Link: https://www.nature.com/articles/s41592-025-02698-z

Reference:

1. Fan, Jingtao, et al. Video-rate imaging of biological dynamics at centimetre scale and micrometre resolution. Nat. Photonics 13, 809–816 (2019).

2. Wu, Jiamin, et al. Iterative tomography with digital adaptive optics permits hour-long intravital observation of 3D subcellular dynamics at millisecond scale. Cell, 184, 3318-3332 (2021).

3. Wu, Jiamin, et al. An integrated imaging sensor for aberration-corrected 3D photography. Nature, 612, 62-71 (2022).

4. Lu, Zhi, et al. Virtual-scanning light-field microscopy for robust snapshot high-resolution volumetric imaging. Nat Methods 20, 735–746 (2023).

5. Lu, Zhi, et al. Long-term intravital subcellular imaging with confocal scanning light-field microscopy. Nat Biotechnol 43, 569–580 (2025).

6. Li, Xinyang, et al. Reinforcing neuron extraction and spike inference in calcium imaging using deep self-supervised denoising. Nat Methods 18, 1395–1400 (2021).

7. Li, Xinyang, et al. Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat Biotechnol 41, 282–292 (2023).

8. Zhang, Guoxun, et al. Bio-friendly long-term subcellular dynamic recording by self-supervised image enhancement microscopy. Nat Methods 20, 1957–1970 (2023).

9. Zhao, Zhifeng, et al. Two-photon synthetic aperture microscopy for minimally invasive fast 3D imaging of native subcellular behaviors in deep tissue. Cell 186, 2475-2491, (2023).

10. Zhang, Yuanlong, et al. Long-term mesoscale imaging of 3D intercellular dynamics across a mammalian organ. Cell 187, 6104-6122 (2024).

11. Strack, R. A leap for mesoscale imaging. Nat Methods 21, 1973-1973 (2024).

12. Li, Xinyang, et al. Challenges and opportunities in bioimage analysis. Nat Methods 20, 958–961 (2023).