Better Fine-Tuning via Instance Weighting for Text Classification


  • Zhi Wang Independent Researcher
  • Wei Bi Tencent AI Lab
  • Yan Wang Tencent AI Lab
  • Xiaojiang Liu Tencent AI Lab



Transfer learning for deep neural networks has achieved great success in many text classification applications. A simple yet effective transfer learning method is to fine-tune the pretrained model parameters. Previous fine-tuning works mainly focus on the pre-training stage and investigate how to pretrain a set of parameters that can help the target task most. In this paper, we propose an Instance Weighting based Finetuning (IW-Fit) method, which revises the fine-tuning stage to improve the final performance on the target domain. IW-Fit adjusts instance weights at each fine-tuning epoch dynamically to accomplish two goals: 1) identify and learn the specific knowledge of the target domain effectively; 2) well preserve the shared knowledge between the source and the target domains. The designed instance weighting metrics used in IW-Fit are model-agnostic, which are easy to implement for general DNN-based classifiers. Experimental results show that IW-Fit can consistently improve the classification accuracy on the target domain.




How to Cite

Wang, Z., Bi, W., Wang, Y., & Liu, X. (2019). Better Fine-Tuning via Instance Weighting for Text Classification. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 7241-7248.



AAAI Technical Track: Natural Language Processing