This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:


peer-reviewed publication

trusted source


AI depicts 3D social interactions between animals

AI depicts 3D social interactions between animals
The architecture of SBeA. Credit: Nature Machine Intelligence (2024). DOI: 10.1038/s42256-023-00776-5

Accurate quantification of multi-animal behavior plays a pivotal role in unraveling the intricacies of animal social interactions with far-reaching applications in neuroscience and ecology.

In a study published in Nature Machine Intelligence, researchers from the Brain Cognition and Brain Disease Institute (BCBDI) of the Shenzhen Institute of Advanced Technology of the Chinese Academy of Sciences have proposed a few-shot learning (AI) framework, the Social Behavior Atlas (SBeA), for multi-animal three-dimensional (3D) social pose estimation, identification, and behavior embedding.

While recent advances in deep learning methods such as multi-animal DeepLabCut, SLEAP, and SIPEC have improved the accessibility of quantifying high-dimensional social behaviors in animals, including pose estimation, identity recognition, and behavior classification, the applications of these methods are restricted by the availability of insufficiently annotated datasets.

In this study, researchers developed a continuously occluded copy-paste algorithm (COCA) as a universal data augmenter to reduce data annotation to about 400 frames in the multi-animal pose estimation step. The number of required data is equivalent to single-animal annotations. COCA promotes SBeA to reach higher performance than state-of-the-art methods. Additionally, SBeA can reconstruct the 3D poses of social animals combined with the camera array.

The proposed bidirectional transfer learning in SBeA can recognize each animal's identity during without the need for manual annotations. This resolves the problem of perplexing identity recognition for animals with similar appearances, even for professional human annotators.

The 3D social poses with identities were further decomposed and clustered by the unsupervised social behavior classification of SBeA, which classifies social behavior without predefined categories.

SBeA helps researchers identify undefined subtle social behavior modules in Shank3B , an used to simulate autism spectrum disorders. It indicates the existence of unknown neural modulation mechanisms behind subtle social behaviors. In addition to mice, SBeA effectively identifies subtle social behavior in other species such as birds and dogs. Neuroscience and ecology would benefit from the accurate animal quantification provided by SBeA.

More information: Yaning Han et al, Multi-animal 3D social pose estimation, identification and behaviour embedding with a few-shot learning framework, Nature Machine Intelligence (2024). DOI: 10.1038/s42256-023-00776-5

Journal information: Nature Machine Intelligence

Citation: AI depicts 3D social interactions between animals (2024, January 9) retrieved 18 June 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Time to get social: Tracking animals with deep learning


Feedback to editors