Connecting the Dots Using Contextual Information Hidden in Text and Images

Authors

  • Md Abdul Kader The University of Texas at El Paso
  • Sheikh Naim The University of Texas at El Paso
  • Arnold Boedihardjo U. S. Army Corps of Engineers, Alexandria, VA
  • M. Shahriar Hossain The University of Texas at El Paso

DOI:

https://doi.org/10.1609/aaai.v30i1.9930

Abstract

Creation of summaries of events of interest from multitude of unstructured data is a challenging task commonly faced by intelligence analysts while seeking increased situational awareness. This paper proposes a framework called Storyboarding that leverages unstructured text and images to explain events as sets of sub-events. The framework first generates a textual context for each human face detected from images and then builds a chain of coherent documents where two consecutive documents of the chain contain a common theme as well as a context. Storyboarding helps analysts quickly narrow down large number of possibilities to a few significant ones for further investigation. Empirical studies on Wikipedia documents, images and news articles show that Storyboarding is able to provide deeper insights on events of interests.

Downloads

Published

2016-03-05

How to Cite

Kader, M. A., Naim, S., Boedihardjo, A., & Hossain, M. S. (2016). Connecting the Dots Using Contextual Information Hidden in Text and Images. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). https://doi.org/10.1609/aaai.v30i1.9930