We focus on the challenging problem of efficient mouse 3D pose estimation based on static images, and especially single depth images. We introduce an approach to discriminatively train the split nodes of trees in random forest to improve their performance on estimation of 3D joint positions of mouse. Our algorithm is capable of working with different types of rodents and with different types of depth cameras and imaging setups. In particular, it is demonstrated in this paper that when a top-mounted depth camera is combined with a bottom-mounted color camera, the final system is capable of delivering full-body pose estimation including four limbs and the paws. Empirical examinations on synthesized and real-world depth images confirm the applicability of our approach on mouse pose estimation, as well as the closely related task of part-based labeling of mouse.
@cite_18: A quantitative description of animal social behaviour is informative for behavioural biologists and clinicians developing drugs to treat social disorders. Social interaction in a group of animals has been difficult to measure because behaviour develops over long periods of time and requires tedious manual scoring, which is subjective and often non-reproducible. Computer-vision systems with the ability to measure complex social behaviour automatically would have a transformative impact on biology. Here, we present a method for tracking group-housed mice individually as they freely interact over multiple days. Each mouse is bleach-marked with a unique fur pattern. The patterns are automatically learned by the tracking software and used to infer identities. Trajectories are analysed to measure behaviour as it develops over days, beyond the range of acute experiments. We demonstrate how our system may be used to study the development of place preferences, associations and social relationships by tracking four mice continuously for five days. Our system enables accurate and reproducible characterisation of wild-type mouse social behaviour and paves the way for high-throughput long-term observation of the effects of genetic, pharmacological and environmental manipulations.
@cite_14: We describe software, MiceProfiler, for automatic tracking of two interacting mice and analysis of their social interactions without the need of animal tagging. The program allows the identification of key elements that trigger social contact in different mouse strains.
@cite_7: Quantification of the behavioural phenotype of animals within a group requires simultaneous position and identity tracking of multiple individuals. Here the authors report an automated tracking system that combines video- and RFID-tracking data and allows behavioural phenotyping of uniquely identified group-living animals.
@cite_12: We present a novel method for analyzing social behavior. Continuous videos are segmented into action ‘bouts’ by building a temporal context model that combines features from spatio-temporal energy and agent trajectories. The method is tested on an unprecedented dataset of videos of interacting pairs of mice, which was collected as part of a state-of-the-art neurophysiological study of behavior. The dataset comprises over 88 hours (8 million frames) of annotated videos. We find that our novel trajectory features, used in a discriminative framework, are more informative than widely used spatio-temporal features; furthermore, temporal context plays an important role for action recognition in continuous videos. Our approach may be seen as a baseline method on this dataset, reaching a mean recognition rate of 61.2 compared to the expert's agreement rate of about 70 .
There has been substantial progress in mouse tracking using color cameras. Many commercial software used for studying social interaction of mice, such as HomeCageScan, EthoVision, AnyMaze, PhenoTracker and MiceProfiler @cite_14 use this technique to track up to two mice from color videos. A common issue with tracking is that it is highly sensitive to small color or texture perturbations over time. This often gives rise to failures, which makes a manual re-initialization of the system necessary and inevitable if used to track for long durations. There has been some recent progress in long-duration tracking, such as @cite_12 @cite_7 @cite_18 , which can work robustly without such manual intervention. This is typically achieved with the help of invasive methods such as attaching RFID chips or applying dedicated fur dyes to maintain identities reliably over time.