The normal functioning of living organisms relies on a series of spatiotemporally coordinated cellular and subcellular activities. Observing and recording these phenomena is the first step to understanding them. Fluorescence microscopy, combined with the development of fluorescent probes, provides scientists with a tool for high spatiotemporal resolution and molecular specificity. Advances in fluorescence imaging have allowed us to resolve the mechanisms of life activities with unprecedented resolution, from the interactions of organelles at the nanoscale, to the footprints of cells during embryonic development, and whole-brain neural activity synchronized with specific behaviors.

A fundamental challenge in fluorescence imaging is the photon shot noise caused by the inherent randomness in photon detection. This randomness is dictated by the quantum nature of light. Essentially, all measurement processes follow the laws of quantum mechanics, with the precision limit known as the photon noise limit. Theoretically, this limit sets an upper bound on the imaging signal-to-noise ratio (SNR). Practically, inherent photon noise increases measurement uncertainty, reducing image quality and limiting resolution, speed, and sensitivity. A 2021 Nature article titled “A shout-out for noise-reduction tools” highlighted that all actual measurement images contain noise, calling for the development of more accurate, reliable, and efficient denoising methods to advance fundamental scientific research.

Breakthrough in Real-Time Denoising

On September 15, 2022, Nature Biotechnology published a paper titled “Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit” by Dai Qionghai’s research group from Tsinghua University.

Following their 2021 Nature Methods publication on the DeepCAD self-supervised denoising framework for neural calcium imaging, the team further optimized this method in terms of network structure and hardware deployment. They enhanced denoising performance and reliability, reduced model size by 16 times, decreased memory consumption by 27%, and shortened processing time by 20 times. They also designed a multithreaded scheduling method to integrate with microscope hardware, achieving real-time denoising on a two-photon microscope (referred to as DeepCAD-RT). This method allows for real-time, ultra-sensitive fluorescence imaging even when the number of detectable fluorescent photons is reduced tenfold. DeepCAD-RT has been applied to various model animals, such as mice, zebrafish, and fruit flies, demonstrating strong denoising performance in observing neuronal calcium dynamics, immune cell migration, neurotransmitter release, and other life processes.

Figure 1: Real-time denoising on a two-photon microscope achieved through model compression and deployment optimization by DeepCAD-RT

In neuroscience, various model animals are used for experimental research. Developing a denoising method applicable to different model animals and cell structures will greatly aid neural imaging data analysis. The authors captured calcium activity in mouse cortical dendritic spines, zebrafish tectal neurons, and fruit fly mushroom body neurons at different resolutions, including low-SNR raw data and synchronized data with tenfold fluorescent photons. After using DeepCAD-RT to remove noise from the low-SNR data, they found that the SNR significantly improved. Neuronal activity previously buried in noise could be accurately restored, with SNR surpassing that of synchronized data with tenfold photons. Various quantitative metrics demonstrated DeepCAD-RT’s compatibility with different model animals and neural structures.

Figure 2: DeepCAD-RT applied to calcium imaging experiments across various models and neural structures

Observing Immune Cell Migration

Immune cell migration is crucial for brain injury repair. High excitation power causes photobleaching, phototoxicity, and tissue damage, disrupting normal immune responses. Observing immune cell migration requires very low excitation power, but ordinary two-photon microscopes have too low SNR under low power to reveal the complete 3D migration process. To solve this, the authors captured the 3D migration of neutrophils in the mouse cortex after acute brain injury using a two-photon microscope at low power, then denoised the images with DeepCAD-RT. Quantitative analysis showed that DeepCAD-RT effectively removed noise, restoring signals buried in the noise and making immune cell structure and migration dynamics clearly visible. This technology enabled the observation of neutrophils dispersing from the injury site after inflammation.

Figure 3: Observing immune cell 3D migration under ultra-low excitation power with DeepCAD-RT

Enhancing Cell Segmentation Accuracy

In studying cell migration, cell segmentation, which provides information on cell number, morphology, and distribution, is often the first step in downstream biological analysis. DeepCAD-RT’s exceptional denoising capability significantly improves segmentation accuracy, aiding researchers in drawing more accurate biological conclusions. Using neutrophil segmentation as an example, the authors employed advanced segmentation methods, Cellpose and Stardist, on both pre- and post-denoised data. The results showed that data denoised by DeepCAD-RT had significantly clearer cell boundaries, greatly enhancing segmentation effectiveness.

Figure 4: DeepCAD-RT denoised data significantly improves cell segmentation accuracy

Monitoring Neurotransmitter Dynamics

With the expanding family of fluorescent probes, neuronal activity probes have evolved from classic calcium probes to include dopamine, glutamate, and acetylcholine probes. In the nervous system, ATP (adenosine triphosphate) serves as an energy molecule and a key signaling molecule. Using the new ATP fluorescent probe GRABATP1.0 developed by Professor Li Yulong’s lab at Peking University, the authors studied the 3D release of ATP around brain tissue injury sites. To avoid experimental bias from phototoxicity and tissue damage caused by excitation, they significantly reduced the excitation power of the two-photon microscope and continuously recorded ATP activity for an hour. Due to low SNR under low power, they used DeepCAD-RT to denoise the experimental data. The results showed that the denoised data revealed more subtle ATP release activities, providing more complete statistical conclusions on release location, timing, and 3D geometry of release sites.

Figure 5: Monitoring 3D dynamic activities of neurotransmitters in the cortex with DeepCAD-RT

Potential Applications and Advancements

As a powerful denoising technology across various model animals and life processes, DeepCAD-RT has the potential to be applied in more imaging scenarios and promote the development of various imaging technologies. For example, when imaging deep tissues, scattering and absorption significantly reduce excitation power, limiting SNR. Light-field microscopy can achieve rapid 3D imaging of living samples but inherent imaging noise complicates reconstruction. Structured illumination microscopy can achieve super-resolution imaging beyond the diffraction limit, but noise similarly limits image reconstruction. By performing the most advanced denoising, DeepCAD-RT helps solve these challenges.

Paper link: https://www.nature.com/articles/s41587-022-01450-8

DeepCAD-RT homepage: https://cabooster.github.io/DeepCAD-RT/

Code link: https://github.com/cabooster/DeepCAD-RT