ELMA: Energy-Based Learning for Multi-Agent Activity Forecasting
Keywords:Computer Vision (CV)
AbstractThis paper describes an energy-based learning method that predicts the activities of multiple agents simultaneously. It aims to forecast both upcoming actions and paths of all agents in a scene based on their past activities, which can be jointly formulated by a probabilistic model over time. Learning this model is challenging because: 1) it has a large number of time-dependent variables that must scale with the forecast horizon and the number of agents; 2) distribution functions have to contain multiple modes in order to capture the spatio-temporal complexities of each agent's activities. To address these challenges, we put forth a novel Energy-based Learning approach for Multi-Agent activity forecasting (ELMA) to estimate this complex model via maximum log-likelihood estimation. Specifically, by sampling from a sequence of factorized marginalized multi-model distributions, ELMA generates most possible future actions efficiently. Moreover, by graph-based representations, ELMA also explicitly resolves the spatio-temporal dependencies of all agents' activities in a single pass. Our experiments on two large-scale datasets prove that ELMA outperforms recent leading studies by an obvious margin.
How to Cite
Li, Y., Wang, P., Chen, L., Wang, Z., & Chan, C.-Y. (2022). ELMA: Energy-Based Learning for Multi-Agent Activity Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 36(2), 1482-1490. https://doi.org/10.1609/aaai.v36i2.20038
AAAI Technical Track on Computer Vision II