Title: Automating Image Analysis by Annotating Landmarks with Deep Neural Networks,
Abstract: Image and video analysis is often a crucial step in the study of animal
behavior and kinematics. Often these analyses require that the position of one
or more animal landmarks are annotated (marked) in numerous images. The process
of annotating landmarks can require a significant amount of time and tedious
labor, which motivates the need for algorithms that can automatically annotate
landmarks. In the community of scientists that use image and video analysis to
study the 3D flight of animals, there has been a trend of developing more
automated approaches for annotating landmarks, yet they fall short of being
generally applicable. Inspired by the success of Deep Neural Networks (DNNs) on
many problems in the field of computer vision, we investigate how suitable DNNs
are for accurate and automatic annotation of landmarks in video datasets
representative of those collected by scientists studying animals.
Our work shows, through extensive experimentation on videos of hawkmoths,
that DNNs are suitable for automatic and accurate landmark localization. In
particular, we show that one of our proposed DNNs is more accurate than the
current best algorithm for automatic localization of landmarks on hawkmoth
videos. Moreover, we demonstrate how these annotations can be used to
quantitatively analyze the 3D flight of a hawkmoth. To facilitate the use of
DNNs by scientists from many different fields, we provide a self contained
explanation of what DNNs are, how they work, and how to apply them to other
datasets using the freely available library Caffe and supplemental code that we
provide.