Investigation into Training Dynamics of Learned Optimizers (Student Abstract)
DOI:
https://doi.org/10.1609/aaai.v38i21.30514Keywords:
Optimization, Deep Learning, Applications Of AIAbstract
Modern machine learning heavily relies on optimization, and as deep learning models grow more complex and data-hungry, the search for efficient learning becomes crucial. Learned optimizers disrupt traditional handcrafted methods such as SGD and Adam by learning the optimization strategy itself, potentially speeding up training. However, the learned optimizers' dynamics are still not well understood. To remedy this, our work explores their optimization trajectories from the perspective of network architecture symmetries and proposed parameter update distributions.Downloads
Published
2024-03-24
How to Cite
Sobotka, J., & Šimánek, P. (2024). Investigation into Training Dynamics of Learned Optimizers (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 38(21), 23657-23658. https://doi.org/10.1609/aaai.v38i21.30514
Issue
Section
AAAI Student Abstract and Poster Program