Invariant Random Forest: Tree-Based Model Solution for OOD Generalization
DOI:
https://doi.org/10.1609/aaai.v38i12.29283Keywords:
ML: Transfer, Domain Adaptation, Multi-Task Learning, ML: Ensemble MethodsAbstract
Out-Of-Distribution (OOD) generalization is an essential topic in machine learning. However, recent research is only focusing on the corresponding methods for neural networks. This paper introduces a novel and effective solution for OOD generalization of decision tree models, named Invariant Decision Tree (IDT). IDT enforces a penalty term with regard to the unstable/varying behavior of a split across different environments during the growth of the tree. Its ensemble version, the Invariant Random Forest (IRF), is constructed. Our proposed method is motivated by a theoretical result under mild conditions, and validated by numerical tests with both synthetic and real datasets. The superior performance compared to non-OOD tree models implies that considering OOD generalization for tree models is absolutely necessary and should be given more attention.Downloads
Published
2024-03-24
How to Cite
Liao, Y., Wu, Q., & Yan, X. (2024). Invariant Random Forest: Tree-Based Model Solution for OOD Generalization. Proceedings of the AAAI Conference on Artificial Intelligence, 38(12), 13772-13781. https://doi.org/10.1609/aaai.v38i12.29283
Issue
Section
AAAI Technical Track on Machine Learning III