UnFlow: Unsupervised Learning of Optical Flow With a Bidirectional Census Loss

Authors

  • Simon Meister TU Darmstadt
  • Junhwa Hur TU Darmstadt
  • Stefan Roth TU Darmstadt

DOI:

https://doi.org/10.1609/aaai.v32i1.12276

Keywords:

Motion, Deep Learning/Neural Networks, Unsupervised Learning

Abstract

In the era of end-to-end deep learning, many advances in computer vision are driven by large amounts of labeled data. In the optical flow setting, however, obtaining dense per-pixel ground truth for real scenes is difficult and thus such data is rare. Therefore, recent end-to-end convolutional networks for optical flow rely on synthetic datasets for supervision, but the domain mismatch between training and test scenarios continues to be a challenge. Inspired by classical energy-based optical flow methods, we design an unsupervised loss based on occlusion-aware bidirectional flow estimation and the robust census transform to circumvent the need for ground truth flow. On the KITTI benchmarks, our unsupervised approach outperforms previous unsupervised deep networks by a large margin, and is even more accurate than similar supervised methods trained on synthetic datasets alone. By optionally fine-tuning on the KITTI training data, our method achieves competitive optical flow accuracy on the KITTI 2012 and 2015 benchmarks, thus in addition enabling generic pre-training of supervised networks for datasets with limited amounts of ground truth.

Downloads

Published

2018-04-27

How to Cite

Meister, S., Hur, J., & Roth, S. (2018). UnFlow: Unsupervised Learning of Optical Flow With a Bidirectional Census Loss. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). https://doi.org/10.1609/aaai.v32i1.12276