LCD: Learned Cross-Domain Descriptors for 2D-3D Matching


  • Quang-Hieu Pham Singapore University of Technology and Design
  • Mikaela Angelina Uy Stanford University
  • Binh-Son Hua The University of Tokyo
  • Duc Thanh Nguyen Deakin University
  • Gemma Roig Geothe University of Frankfrut am Main
  • Sai-Kit Yeung Hong Kong University of Science and Technology



In this work, we present a novel method to learn a local cross-domain descriptor for 2D image and 3D point cloud matching. Our proposed method is a dual auto-encoder neural network that maps 2D and 3D input into a shared latent space representation. We show that such local cross-domain descriptors in the shared embedding are more discriminative than those obtained from individual training in 2D and 3D domains. To facilitate the training process, we built a new dataset by collecting ≈ 1.4 millions of 2D-3D correspondences with various lighting conditions and settings from publicly available RGB-D scenes. Our descriptor is evaluated in three main experiments: 2D-3D matching, cross-domain retrieval, and sparse-to-dense depth estimation. Experimental results confirm the robustness of our approach as well as its competitive performance not only in solving cross-domain tasks but also in being able to generalize to solve sole 2D and 3D tasks. Our dataset and code are released publicly at




How to Cite

Pham, Q.-H., Uy, M. A., Hua, B.-S., Nguyen, D. T., Roig, G., & Yeung, S.-K. (2020). LCD: Learned Cross-Domain Descriptors for 2D-3D Matching. Proceedings of the AAAI Conference on Artificial Intelligence, 34(07), 11856-11864.



AAAI Technical Track: Vision