Synthetic Data Generation for Deep Learning-based Semantic Segmentation

Abstract

The semantic segmentation of a scene is one of the basic components towards the total understanding of this scene that make up a robotic perception system. Currently, systems based on deep learning, specifically convolutional networks, dominate the state of the art with highly accurate results. However, these systems rely on datasets of unprecedented scale and variability in order to properly generalize into the potentially infinite number of situations in which they can be deployed. Current datasets often have problems in achieving this scale and variability as they rely on human operators both for the capture of the data itself and for its labelling, which is essential for this type of supervised learning techniques. The high cost in time and resources of this task makes it difficult to obtain large-scale and highly representative data sets for specific situations. In this work we propose the exploration of photorealistic synthetic data as a source to train new systems, to improve the capacity of generalization of those already trained with real data or to facilitate training when a small amount of them is available. To do this we will resort to Unreal Engine 4 to create UnrealROX1 with the objective of generating an extremely photorealistic data set. We will implement a series of tools to generate this data by creating a simulator capable of doing this work.