Bayesian Maximum Margin Principal Component Analysis


  • Changying Du Chinese Academy of Sciences
  • Shandian Zhe Purdue University
  • Fuzhen Zhuang Chinese Academy of Sciences
  • Yuan Qi Purdue University
  • Qing He Chinese Academy of Sciences
  • Zhongzhi Shi Chinese Academy of Sciences


Supervised dimensionality reduction, Principal Component Analysis (PCA), Maximum margin principle, Variational Bayesian


Supervised dimensionality reduction has shown great advantages in finding predictive subspaces. Previous methods rarely consider the popular maximum margin principle and are prone to overfitting to usually small training data, especially for those under the maximum likelihood framework. In this paper, we present a posterior-regularized Bayesian approach to combine Principal Component Analysis (PCA) with the max-margin learning. Based on the data augmentation idea for max-margin learning and the probabilistic interpretation of PCA, our method can automatically infer the weight and penalty parameter of max-margin learning machine, while finding the most appropriate PCA subspace simultaneously under the Bayesian framework. We develop a fast mean-field variational inference algorithm to approximate the posterior. Experimental results on various classification tasks show that our method outperforms a number of competitors.




How to Cite

Du, C., Zhe, S., Zhuang, F., Qi, Y., He, Q., & Shi, Z. (2015). Bayesian Maximum Margin Principal Component Analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1). Retrieved from



Main Track: Novel Machine Learning Algorithms