Rule Ensemble Learning Using Hierarchical Kernels in Structured Output Spaces

Authors

  • Naveen Nair IITB-Monash Research Academy, IIT Bombay, Monash University.
  • Amrita Saha IIT Bombay
  • Ganesh Ramakrishnan IIT Bombay
  • Shonali Krishnaswamy Institute for Infocomm Research (I2R), Singapore; Monash University, Australia

DOI:

https://doi.org/10.1609/aaai.v26i1.8284

Keywords:

activity recognition, hidden markov models, hierarchical kernels, rule learning, structured output spaces, support vector machines.

Abstract

The goal in Rule Ensemble Learning (REL) is simultaneous discovery of a small set of simple rules and their optimal weights that lead to good generalization. Rules are assumed to be conjunctions of basic propositions concerning the values taken by the input features. It has been shown that rule ensembles for classification can be learnt optimally and efficiently using hierarchical kernel learning approaches that explore the exponentially large space of conjunctions by exploiting its hierarchical structure. The regularizer employed penalizes large features and thereby selects a small set of short features. In this paper, we generalize the rule ensemble learning using hierarchical kernels (RELHKL) framework to multi class structured output spaces. We build on the StructSVM model for sequence prediction problems and employ a ρ-norm hierarchical regularizer for observation features and a conventional 2-norm regularizer for state transition features. The exponentially large feature space is searched using an active set algorithm and the exponentially large set of constraints are handled using a cutting plane algorithm. The approach can be easily extended to other structured output problems. We perform experiments on activity recognition datasets which are prone to noise, sparseness and skewness. We demonstrate that our approach outperforms other approaches.

Downloads

Published

2021-09-20

How to Cite

Nair, N., Saha, A., Ramakrishnan, G., & Krishnaswamy, S. (2021). Rule Ensemble Learning Using Hierarchical Kernels in Structured Output Spaces. Proceedings of the AAAI Conference on Artificial Intelligence, 26(1), 1061-1067. https://doi.org/10.1609/aaai.v26i1.8284

Issue

Section

AAAI Technical Track: Machine Learning