We focus on the challenging problem of efficient mouse 3D pose estimation based on static images, and especially single depth images. We introduce an approach to discriminatively train the split nodes of trees in random forest to improve their performance on estimation of 3D joint positions of mouse. Our algorithm is capable of working with different types of rodents and with different types of depth cameras and imaging setups. In particular, it is demonstrated in this paper that when a top-mounted depth camera is combined with a bottom-mounted color camera, the final system is capable of delivering full-body pose estimation including four limbs and the paws. Empirical examinations on synthesized and real-world depth images confirm the applicability of our approach on mouse pose estimation, as well as the closely related task of part-based labeling of mouse.
@cite_24: Abstract We herein introduce an automated three-dimensional (3D) locomotion tracking and pose reconstruction system for rodents with superior robustness, rapidity, reliability, resolution, simplicity, and cost. An off-the-shelf composite infrared (IR) range camera was adopted to grab high-resolution depth images (640 × 480 × 2048 pixels at 20 Hz) in our system for automated behavior analysis. For the inherent 3D structure of the depth images, we developed a compact algorithm to reconstruct the locomotion and body behavior with superior temporal and solid spatial resolution. Since the range camera operates in the IR spectrum, interference from the visible light spectrum did not affect the tracking performance. The accuracy of our system was 98.1 ± 3.2 . We also validated the system, which yielded strong correlation with automated and manual tracking. Meanwhile, the system replicates a detailed dynamic rat model in virtual space, which demonstrates the movements of the extremities of the body and locomotion in detail on varied terrain.
@cite_12: We present a novel method for analyzing social behavior. Continuous videos are segmented into action ‘bouts’ by building a temporal context model that combines features from spatio-temporal energy and agent trajectories. The method is tested on an unprecedented dataset of videos of interacting pairs of mice, which was collected as part of a state-of-the-art neurophysiological study of behavior. The dataset comprises over 88 hours (8 million frames) of annotated videos. We find that our novel trajectory features, used in a discriminative framework, are more informative than widely used spatio-temporal features; furthermore, temporal context plays an important role for action recognition in continuous videos. Our approach may be seen as a baseline method on this dataset, reaching a mean recognition rate of 61.2 compared to the expert's agreement rate of about 70 .
@cite_8: Numerous ‘non-maximum suppression’ (NMS) post-processing schemes have been proposed for merging multiple independent object detections. We propose a generalization of NMS beyond bounding boxes to merge multiple pose estimates in a single frame. The final estimates are centroids rather than medoids as in standard NMS, thus being more accurate than any of the individual candidates. Using the same mathematical framework, we extend our approach to the multi-frame setting, merging multiple independent pose estimates across space and time and outputting both the number and pose of the objects present in a scene. Our approach sidesteps many of the inherent challenges associated with full tracking (e.g. objects entering leaving a scene, extended periods of occlusion, etc.). We show its versatility by applying it to two distinct state-of-the-art pose estimation algorithms in three domains: human bodies, faces and mice. Our approach improves both detection accuracy (by helping disambiguate correspondences) as well as pose estimation quality and is computationally efficient.
The recent advance of commodity depth imaging opens the door to drastic progress in this direction. These depth cameras are currently based on either structured illumination or time-of-flight (ToF) technologies. The most related work in the field @cite_24 is a low-res body pose estimation that involves the 3D location and orientation of the entire mouse based an overhead Primesense depth camera. Our approach may be viewed as the next step to reveal high-resolution details of 3D mouse full-body skeleton. Note the mice dataset of @cite_12 has also been studied in @cite_8 for a similar low-resolution pose estimation purpose in 2D based on gray-scale images.