Effects of Dataset Bias on Conditional Generative Adversarial Networks for Urban Scene Understanding

Abstract

Despite the many recent developments of deep learning models in urban scenes, dataset bias still poses a significant barrier to effectively adapting these models to other domains, and eventually, the real world. Furthermore, data collection and ground truth annotation are difficult and expensive tasks due to the diverse and busy nature of urban scenes. Research on developing domain adaptation and image-to-image translation frameworks has shown the potential of using synthetic data to combat this problem. This work investigates the out-of-domain behavior of pix2pixHD, a conditional generative adversarial network trained to convert pixel-wise semantic labels to photorealistic images. It identifies a performance gap between Cityscapes and Crosscity, two geographically diverse urban scenes datasets, and experiments with various importance factors to test their effect on overall dataset bias. Results provide a feature-based explanation for model behavior by showing how differences in importance factors like class composition and image perspective can lead to capture bias. In addition, the conclusions support the current literature in highlighting the discriminative role of buildings in cities and offer guidance for future model adaption and robust data collection.