As an initial assessment, over 480,000 labeled virtual images of normal highway driving were readily generated in Grand Theft Auto V's virtual environment. Using these images, a CNN was trained to detect following distance to cars objects ahead, lane markings, and driving angle (angular heading relative to lane centerline): all variables necessary for basic autonomous driving. Encouraging results were obtained when tested on over 50,000 labeled virtual images from substantially different GTA-V driving environments. This initial assessment begins to define both the range and scope of the labeled images needed for training as well as the range and scope of labeled images needed for testing the definition of boundaries and limitations of trained networks. It is the efficacy and flexibility of a "GTA-V"-like virtual environment that is expected to provide an efficient well-defined foundation for the training and testing of Convolutional Neural Networks for safe driving. Additionally, described is the Princeton Virtual Environment (PVE) for the training, testing and enhancement of safe driving AI, which is being developed using the video-game engine Unity. PVE is being developed to recreate rare but critical corner cases that can be used in re-training and enhancing machine learning models and understanding the limitations of current self driving models. The Florida Tesla crash is being used as an initial reference.
@cite_5: This article describes the implementation of the cooperative project "Autonomous Driving" which had been set up for the automation of vehicle test drives on the VOLKSWAGEN approved ground. As part of the concept, ordinary mass-produced cars can be transformed into automatically driven cars within a few hours by skilled persons. In the project, two test vehicles were equipped with sensors, computers and actuators (driving robots). They can be driven by a robot while the sensor system observes the entire surroundings to be able to react on obstacles. Furthermore, the vehicle control can be overruled by an electronic copilot in case of an emergency.
@cite_20: Today, visual recognition systems are still rarely employed in robotics applications. Perhaps one of the main reasons for this is the lack of demanding benchmarks that mimic such scenarios. In this paper, we take advantage of our autonomous driving platform to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry SLAM and 3D object detection. Our recording platform is equipped with four high resolution video cameras, a Velodyne laser scanner and a state-of-the-art localization system. Our benchmarks comprise 389 stereo and optical flow image pairs, stereo visual odometry sequences of 39.2 km length, and more than 200k 3D object annotations captured in cluttered scenarios (up to 15 cars and 30 pedestrians are visible per image). Results from state-of-the-art algorithms reveal that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world. Our goal is to reduce this bias by providing challenging benchmarks with novel difficulties to the computer vision community. Our benchmarks are available online at: www.cvlibs.net datasets kitti
Since early on in the development of self-driving technology physical test tracks have been used to evaluate self-driving algorithms @cite_5 . The final benchmark for system's performance will be real world testing, but with rigorous and complex examples of labeled datasets it is possible to evaluate many corner cases. In 2012 released the KITTI dataset which has become the preeminent dataset in testing self-driving technologies. @cite_20 The value in the labeled data stems from the complexity and the number of examples it contains. Just as the KITTI dataset provided immensely useful labeled data to train and test on, it is possible to use a virtual environment to create varied and difficult test cases for self-driving technologies.