Learning Deep &#8467;<sub>0</sub> Encoders

Authors

  • Zhangyang Wang University of Illinois at Urbana-Champaign
  • Qing Ling University of Science and Technology of China
  • Thomas Huang University of Illinois at Urbana-Champaign

DOI:

https://doi.org/10.1609/aaai.v30i1.10198

Keywords:

deep learning, sparse approximation

Abstract

Despite its nonconvex nature, ℓ0 sparse approximation is desirable in many theoretical and application cases. We study the ℓ0 sparse approximation problem with the tool of deep learning, by proposing Deep ℓ0 Encoders. Two typical forms, the ℓ0 regularized problem and the M-sparse problem, are investigated. Based on solid iterative algorithms, we model them as feed-forward neural networks, through introducing novel neurons and pooling functions. Enforcing such structural priors acts as an effective network regularization. The deep encoders also enjoy faster inference, larger learning capacity, and better scalability compared to conventional sparse coding solutions. Furthermore, under task-driven losses, the models can be conveniently optimized from end to end. Numerical results demonstrate the impressive performances of the proposed encoders.

Downloads

Published

2016-03-02

How to Cite

Wang, Z., Ling, Q., & Huang, T. (2016). Learning Deep &#8467;<sub>0</sub> Encoders. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). https://doi.org/10.1609/aaai.v30i1.10198

Issue

Section

Technical Papers: Machine Learning Methods