Dimensionality Reduction Using Bayesian Learning Predictive Subspaces For Supervised And Semi- Supervised Multi-Label Learning

Abstract: For supervised learning problems, dimensionality reduction is generally applied as a pre-processing step. However, Coupled training of dimensionality reduction and classification is proposed previously to improve the prediction performance for single-label problems. In this paper, we first introduce a novel Bayesian method that combines linear dimensionality reduction with linear binary classification for supervised multi-label learning and present a deterministic variational approximation algorithm to learn the proposed probabilistic model. The proposed method is to find intrinsic dimensionality of the projected subspace using automatic relevance determination and to handle semi-supervised learning using a low-density assumption. Our proposed method significantly outperforms combined Bayesian with multiple kernel Fisher discriminate analysis followed by a standard kernel-based learner, especially on low dimensions.

Keywords: Dimensionality reduction, multi-label learning, subspace learning, Bayesian, supervised learning, semi- supervised learning.