Learning Optimal Features via Partial Invariance
DOI:
https://doi.org/10.1609/aaai.v37i6.25875Keywords:
ML: Transfer, Domain Adaptation, Multi-Task Learning, ML: Bayesian Learning, ML: Classification and RegressionAbstract
Learning models that are robust to distribution shifts is a key concern in the context of their real-life applicability. Invariant Risk Minimization (IRM) is a popular framework that aims to learn robust models from multiple environments. The success of IRM requires an important assumption: the underlying causal mechanisms/features remain invariant across environments. When not satisfied, we show that IRM can over-constrain the predictor and to remedy this, we propose a relaxation via partial invariance. In this work, we theoretically highlight the sub-optimality of IRM and then demonstrate how learning from a partition of training domains can help improve invariant models. Several experiments, conducted both in linear settings as well as with deep neural networks on tasks over both language and image data, allow us to verify our conclusions.Downloads
Published
2023-06-26
How to Cite
Choraria, M., Ferwana, I., Mani, A., & Varshney, L. R. (2023). Learning Optimal Features via Partial Invariance. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6), 7175-7183. https://doi.org/10.1609/aaai.v37i6.25875
Issue
Section
AAAI Technical Track on Machine Learning I