Animal tracking is an important approach for understanding social interactions, neural mechanisms, and behavioral decision-making. It is widely used in neuroscience, psychology, pharmacology, disease model research, and many other fields. Achieving precise and efficient multi-animal tracking will strongly promote research across multiple disciplines.

In recent years, with the development of artificial intelligence (AI), animal tracking has become increasingly automated and intelligent. However, existing tracking methods rely on manual annotations for supervised training, which is costly, inefficient, subject to human variability, and difficult to cover the full diversity of animal species, experimental paradigms, and behavioral patterns. How to achieve precise tracking across multiple model animals without manual annotation has become a key problem that urgently needs to be solved in the field of animal tracking.

On May 4, 2026, the research teams of Qionghai Dai, Jiamin Wu, and Xinyang Li from Tsinghua University, together with the team of Ziwei Li from Fudan University, published a research article titled Unsupervised transfer learning enables multi-animal tracking without training annotation in Nature Methods. The study presents UDMT, an unsupervised transfer learning method for precise multi-animal tracking. Without requiring any manually annotated training samples, UDMT can train a tracking model using only the raw input video. It is applicable to multiple model animals and achieves precise and efficient multi-animal tracking under complex conditions, including crowding, occlusion, rapid motion, low contrast, and cross-species experiments.

Figure 1. UDMT achieves precise tracking across multiple model animals

UDMT is based on the principle of bidirectional-consistency tracking. The network first performs forward tracking along the video and then performs backward tracking in the reverse direction. When the network converges, the forward and backward trajectories should completely overlap. Based on this principle, the research team constructs training signals from the temporal continuity of the video itself, allowing the AI model to automatically learn animal appearance features and movement patterns from data without any manually annotated training samples. In addition, the team further developed key techniques including spatiotemporal information aggregation, target localization optimization, identity error correction, and automatic parameter tuning, significantly improving tracking accuracy and stability in complex scenarios. In practical use, researchers only need to specify the animal individuals to be tracked, and UDMT can automatically complete model training and subsequent tracking, outputting the complete movement trajectory of each animal. This method eliminates the heavy burden of manual annotation and parameter tuning, bringing multi-animal tracking into a new stage of unsupervised, efficient, and automated analysis.

Figure 2. Principle of UDMT

Real behavioral experiments often involve complex conditions such as crowding, occlusion, low contrast, cross-species settings, and rapid motion. The research team applied UDMT to crowded high-contrast black mice, low-contrast white mice, cross-species experiments involving rats and mice, and scenarios with complex backgrounds. The results show that UDMT maintains superior accuracy and stability across various scenarios. Compared with existing state-of-the-art supervised learning methods such as DeepLabCut [1], SLEAP [2], idtracker.ai [3], and TRex [4], UDMT requires no manual annotation and achieves higher tracking accuracy across different animal numbers, recording durations, and frame rates.

Figure 3. UDMT outperforms current state-of-the-art supervised methods

Animal behavior is closely related to neural activity, and UDMT is playing an important role in frontier neuroethological research. To investigate the relationship between neural activity and animal behavior, the research team combined UDMT with a head-mounted miniaturized microscope [5] and established a neuroethological recording platform for freely moving mice. While precisely capturing mouse movement trajectories, the platform also records the activity of more than 2,000 neurons in the visual cortex using an advanced head-mounted microscope. The analysis shows that when companion mice are nearby, the overall neuronal firing rate significantly increases; when the mouse moves at a higher speed, neural activity is also significantly enhanced.

Figure 4. UDMT combined with a head-mounted microscope enables neuroethological analysis of freely moving mice

In addition to rodents, behavioral research also widely involves multiple model animals such as insects, worms, and fish. Different animals vary greatly in body size, movement patterns, imaging scale, and behavioral modes, placing higher demands on the generalizability of tracking methods. The research team further extended UDMT to animals including Drosophila, Caenorhabditis elegans, and betta fish, achieving precise continuous tracking of up to 17 animals. These cross-species experiments demonstrate that UDMT does not depend on a single animal type or a specific experimental setup, and can be applied to diverse animal behavioral experiments. The research team has released the Python source code, graphical user interface, behavioral recording data, and detailed tutorials for UDMT, promoting its application in broader scientific research scenarios.

Authors and affiliations:

Academician Qionghai Dai, Associate Professor Jiamin Wu, and Assistant Professor Xinyang Li from Tsinghua University, together with Junior Associate Researcher Ziwei Li from Fudan University, are the co-corresponding authors of this paper. PhD student Yixin Li from Fudan University is the first author. Qi Zhang, Yuanlong Zhang, Jiaqi Fan, Zhi Lu, and Xinhong Xu also participated in the study and made important contributions.

UDMT homepage: https://cabooster.github.io/UDMT/

Code: https://github.com/cabooster/UDMT