Exploiting Depth and Motion Cues for Robust Semantic, Instance-level Segmentation and Tracking

Principal Investigator:

TRI Liaison:

Project Summary

We aim to develop a semantic, instance segmentation method that exposes increased robustness under challenging real-world conditions. Central to our approach is (i) to avoid the vulnerable, modularized process in current state-of-the-art region-proposal based methods and (ii) to exploit the powerful cues of depth and motion.

Robust, semantic, instance segmentation is fundamental for both, future intelligent vehicles and robots. It provides an accurate understanding of the environment under diverse and challenging conditions. Thereby, it enables safe and robust decision-making. Our approach leverages characteristics typical of both, autonomous cars and robots. It supports the themes of Self-Driving Cars and Robotics and Smart Environments.

Research Goals

Reformulate instance segmentation as one-stage regression problem in 3D to ease training, improve robustness and efficiency
Exploiting motion cues and assuming rigidity of objects to further increase robustness
Tracking semantically-annotated objects over a long time horizon to maintain coherent semantic segmentation over time