A General Class of Transfer Learning Regression without Implementation Cost

Authors

  • Shunya Minami The Graduate University for Advanced Studies
  • Song Liu University of Bristol
  • Stephen Wu The Institute of Statistical Mathematics The Graduate University for Advanced Studies
  • Kenji Fukumizu The Institute of Statistical Mathematics The Graduate University for Advanced Studies
  • Ryo Yoshida The Institute of Statistical Mathematics The Graduate University for Advanced Studies National Institute for Materials Science

Keywords:

Transfer/Adaptation/Multi-task/Meta/Automated Learning, Classification and Regression

Abstract

We propose a novel framework that unifies and extends existing methods of transfer learning (TL) for regression. To bridge a pretrained source model to the model on a target task, we introduce a density-ratio reweighting function, which is estimated through the Bayesian framework with a specific prior distribution. By changing two intrinsic hyperparameters and the choice of the density-ratio model, the proposed method can integrate three popular methods of TL: TL based on cross-domain similarity regularization, a probabilistic TL using the density-ratio estimation, and fine-tuning of pretrained neural networks. Moreover, the proposed method can benefit from its simple implementation without any additional cost; the regression model can be fully trained using off-the-shelf libraries for supervised learning in which the original output variable is simply transformed to a new output variable. We demonstrate its simplicity, generality, and applicability using various real data applications.

Downloads

Published

2021-05-18

How to Cite

Minami, S., Liu, S., Wu, S., Fukumizu, K., & Yoshida, R. (2021). A General Class of Transfer Learning Regression without Implementation Cost. Proceedings of the AAAI Conference on Artificial Intelligence, 35(10), 8992-8999. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17087

Issue

Section

AAAI Technical Track on Machine Learning III