Covariate-Shift Generalization via Random Sample Weighting


  • Yue He Tsinghua University
  • Xinwei Shen ETH Zurich
  • Renzhe Xu Tsinghua University
  • Tong Zhang The Hong Kong University of Science and Technology
  • Yong Jiang Tsinghua University
  • Wenchao Zou Siemens China
  • Peng Cui Tsinghua University



PEAI: Safety, Robustness & Trustworthiness, ML: Bias and Fairness


Shifts in the marginal distribution of covariates from training to the test phase, named covariate-shifts, often lead to unstable prediction performance across agnostic testing data, especially under model misspecification. Recent literature on invariant learning attempts to learn an invariant predictor from heterogeneous environments. However, the performance of the learned predictor depends heavily on the availability and quality of provided environments. In this paper, we propose a simple and effective non-parametric method for generating heterogeneous environments via Random Sample Weighting (RSW). Given the training dataset from a single source environment, we randomly generate a set of covariate-determining sample weights and use each weighted training distribution to simulate an environment. We theoretically show that under appropriate conditions, such random sample weighting can produce sufficient heterogeneity to be exploited by common invariance constraints to find the invariant variables for stable prediction under covariate shifts. Extensive experiments on both simulated and real-world datasets clearly validate the effectiveness of our method.




How to Cite

He, Y., Shen, X., Xu, R., Zhang, T., Jiang, Y., Zou, W., & Cui, P. (2023). Covariate-Shift Generalization via Random Sample Weighting. Proceedings of the AAAI Conference on Artificial Intelligence, 37(10), 11828-11836.



AAAI Technical Track on Philosophy and Ethics of AI