DeepFollowing: Vision-Based Distance Estimation on Synthetically-Generated Driving Video using 3D Convolution

Abstract

The recent surge in popularity of autonomous driving vehicles among the public, coupled with real roll-outs of self-driving systems, supports continued academic research in all aspects of this space. Modeling the relationship between the driving vehicle and a vehicle in front of it, known as vehicle-following, is a necessary consideration for any autonomous control system. Beyond valuable applications, such as reducing energy costs in the $700B trucking industry (ATA 2016), studying this highly sequential process provides interesting insight into the temporal structure of driving recognition.A simple vehicle-following system can be reduced to the problem of accurately gauging the distance to a leading vehicle (DTLV). The most efficient vehicle-following system would allow vehicles to stream information between one another and coordinate decisions; however, the likelihood of two randomly-paired vehicles possessing the same notification system is low, at least in the foreseeable future. While lidar and radar systems have been popular solutions, particularly in Adaptive Cruise Control applications, they are expensive and not easily scalable. A promising alternative is the application of vision-based distance estimation to the car-following problem.Producing an annotated data set of real vehicle-following footage is laborious and error prone, so realistic images are captured using the virtual driving environment in Grand Theft Auto V (GTA V). The advantage of such an environment is that driving conditions such as weather, road type, and leading vehicle model can be easily varied. Furthermore, true distances are known and do not have to be inferred. This thesis finds that a novel 3D Convolutional Neural Network (3D CNN) estimates DTLV with mean average error of 4.13m on synthetic test data, a 6.8\% improvement over a comparable 2D CNN. This performance improvement appears to be consistent across nearly all studied virtual driving conditions. Furthermore, training on the synthetic data provides qualitatively reasonable estimation on real-world driving data.