Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection

Authors

  • Edward Raff Booz Allen Hamilton Laboratory for Physical Sciences University of Maryland, Baltimore County
  • William Fleshman U.S. Military
  • Richard Zak Booz Allen Hamilton Laboratory for Physical Sciences University of Maryland, Baltimore County
  • Hyrum S. Anderson Microsoft
  • Bobby Filar Elastic
  • Mark McLean Laboratory for Physical Sciences

Keywords:

Applications, Time-Series/Data Streams, (Deep) Neural Network Algorithms

Abstract

Recent works within machine learning have been tackling inputs of ever increasing size, with cyber security presenting sequence classification problems of particularly extreme lengths. In the case of Windows executable malware detection, an input executable could be >=100 MB, which would translate to a time series with T=100,000,000 steps. To date, the closest approach to handling such task is MalConv --- a convolutional neural network capable of processing T=2,000,000 steps. Because the memory used by CNNs is O(T), this has prevented many from processing all executables or further extending the MalConv approach. In this work, we develop a new approach to temporal max pooling that makes the required memory invariant to the sequence length T. This makes MalConv 116x more memory efficient, and up to 25.8x faster to train, while removing the input length restrictions to MalConv. We re-invest these gains into improving the MalConv architecture by developing a new Global Channel Gating design, giving us an attention mechanism capable of learning feature interactions across 100 million time steps in an efficient manner, a capability lacked by the original MalConv approach.

Downloads

Published

2021-05-18

How to Cite

Raff, E., Fleshman, W., Zak, R., Anderson, H. S., Filar, B., & McLean, M. (2021). Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 35(11), 9386-9394. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17131

Issue

Section

AAAI Technical Track on Machine Learning IV