Bayesian Maximum Margin Principal Component Analysis

Authors

  • Changying Du Chinese Academy of Sciences
  • Shandian Zhe Purdue University
  • Fuzhen Zhuang Chinese Academy of Sciences
  • Yuan Qi Purdue University
  • Qing He Chinese Academy of Sciences
  • Zhongzhi Shi Chinese Academy of Sciences

Keywords:

Supervised dimensionality reduction, Principal Component Analysis (PCA), Maximum margin principle, Variational Bayesian

Abstract

Supervised dimensionality reduction has shown great advantages in finding predictive subspaces. Previous methods rarely consider the popular maximum margin principle and are prone to overfitting to usually small training data, especially for those under the maximum likelihood framework. In this paper, we present a posterior-regularized Bayesian approach to combine Principal Component Analysis (PCA) with the max-margin learning. Based on the data augmentation idea for max-margin learning and the probabilistic interpretation of PCA, our method can automatically infer the weight and penalty parameter of max-margin learning machine, while finding the most appropriate PCA subspace simultaneously under the Bayesian framework. We develop a fast mean-field variational inference algorithm to approximate the posterior. Experimental results on various classification tasks show that our method outperforms a number of competitors.

Downloads

Published

2015-02-21

How to Cite

Du, C., Zhe, S., Zhuang, F., Qi, Y., He, Q., & Shi, Z. (2015). Bayesian Maximum Margin Principal Component Analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/9583

Issue

Section

Main Track: Novel Machine Learning Algorithms