Build accurate DNN models requires training on large labeled, context specific datasets, especially those matching the target scenario. We believe advances in wireless localization, working in unison with cameras, can produce automated annotation of targets on images and videos captured in the wild. Using pedestrian and vehicle detection as examples, we demonstrate the feasibility, benefits, and challenges of an automatic image annotation system. Our work calls for new technical development on passive localization, mobile data analytics, and error-resilient ML models, as well as design issues in user privacy policies.
@cite_0: 
@cite_18: As for our newly developed smart camera system, a RFID reader is integrated into a digital camera for recognizing user (Who), place (Where) and object (What) that are associated with an image. When an author's RFID tag is detected after the camera system is turned on, corresponding EPC code compliant user information is obtained to perform authentication. All images taken after this would be stored with this attribute indicating the creator of the images. In case other person's RFID tag is detected afterwards, the system recognizes it as the target object that is to be taken into an image. If the detected RFID tag is related to a place, the system records it as the location context. In case GPS is activated at the location, the GPS reading is stored in geotagging format. Unlike computer vision based image annotation process, this new approach could support various geography sensitive imaging services such as an effective image searching on a typical PC. Such a RFID assisted image annotation system enables a highly efficient image classification, archival and retrieval.
@cite_12: In this work we study the use of 3D hand poses to recognize first-person dynamic hand actions interacting with 3D objects. Towards this goal, we collected RGB-D video sequences comprised of more than 100K frames of 45 daily hand action categories, involving 26 different objects in several hand configurations. To obtain hand pose annotations, we used our own mo-cap system that automatically infers the 3D location of each of the 21 joints of a hand model via 6 magnetic sensors and inverse kinematics. Additionally, we recorded the 6D object poses and provide 3D object models for a subset of hand-object interaction sequences. To the best of our knowledge, this is the first benchmark that enables the study of first-person hand actions with the use of 3D hand poses. We present an extensive experimental evaluation of RGB-D and pose-based action recognition by 18 baselines state-of-the-art approaches. The impact of using appearance features, poses, and their combinations are measured, and the different training testing protocols are evaluated. Finally, we assess how ready the 3D hand pose estimation field is when hands are severely occluded by objects in egocentric views and its influence on action recognition. From the results, we see clear benefits of using hand pose as a cue for action recognition compared to other data modalities. Our dataset and experiments can be of interest to communities of 3D hand pose estimation, 6D object pose, and robotics as well as action recognition.
Automatic Annotation. Some prior works annotate objects or gestures by physically tagging them with RFIDs @cite_18 or magnetic sensors @cite_12 . These require active participation by the target and significantly limit scalability and applicable uses. In contrast, our goal is to scalably produce labeled images of passive targets ( e.g. , people, pets with collars, vehicles that already carry WiFi devices). Our work also differs from automatic image annotation that uses visual features ( e.g. , color, texture, shape) to generate image labels. It requires complex generative models that are hard to build @cite_0 .