Visual Comfort Aware-Reinforcement Learning for Depth Adjustment of Stereoscopic 3D Images

Authors

  • Hak Gu Kim KAIST, Korea EPFL, Switzerland
  • Minho Park KAIST, Korea
  • Sangmin Lee KAIST, Korea
  • Seongyeop Kim KAIST, Korea
  • Yong Man Ro KAIST, Korea

Keywords:

Applications, 3D Computer Vision, Reinforcement Learning

Abstract

Depth adjustment aims to enhance the visual experience of stereoscopic 3D (S3D) images, which accompanied with improving visual comfort and depth perception. For a human expert, the depth adjustment procedure is a sequence of iterative decision making. The human expert iteratively adjusted the depth until he is satisfied with the both levels of visual comfort and the perceived depth. In this work, we present a novel deep reinforcement learning (DRL)-based approach for depth adjustment named VCA-RL (Visual Comfort Aware Reinforcement Learning) to explicitly model human sequential decision making in depth editing operations. We formulate the depth adjustment process as a Markov decision process where actions are defined as camera movement operations to control the distance between the left and right cameras. Our agent is trained based on the guidance of an objective visual comfort assessment metric to learn the optimal sequence of camera movement actions in terms of perceptual aspects in stereoscopic viewing. With extensive experiments and user studies, we show the effectiveness of our VCA-RL model on three different S3D databases.

Downloads

Published

2021-05-18

How to Cite

Kim, H. G., Park, M., Lee, S., Kim, S., & Ro, Y. M. (2021). Visual Comfort Aware-Reinforcement Learning for Depth Adjustment of Stereoscopic 3D Images. Proceedings of the AAAI Conference on Artificial Intelligence, 35(2), 1762-1770. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16270

Issue

Section

AAAI Technical Track on Computer Vision I