HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks

Authors

  • Firoj Alam QCRI, Doha, Qatar
  • Umair Qazi QCRI, Doha, Qatar
  • Muhammad Imran QCRI, Doha, Qatar
  • Ferda Ofli QCRI, Doha, Qatar

DOI:

https://doi.org/10.1609/icwsm.v15i1.18116

Keywords:

Measuring predictability of real world phenomena based on social media, e.g., spanning politics, finance, and health, Text categorization; topic recognition; demographic/gender/age identification, Qualitative and quantitative studies of social media

Abstract

Social networks are widely used for information consumption and dissemination, especially during time-critical events such as natural disasters. Despite its significantly large volume, social media content is often too noisy for direct use in any application. Therefore, it is important to filter, categorize, and concisely summarize the available content to facilitate effective consumption and decision-making. To address such issues automatic classification systems have been developed using supervised modeling approaches, thanks to the earlier efforts on creating labeled datasets. However, existing datasets are limited in different aspects (e.g., size, contains duplicates) and less suitable to support more advanced and data-hungry deep learning models. In this paper, we present a new large-scale dataset with ~77K human-labeled tweets, sampled from a pool of ~24 million tweets across 19 disaster events that happened between 2016 and 2019. Moreover, we propose a data collection and sampling pipeline, which is important for social media data sampling for human annotation. We report multiclass classification results using classic and deep learning (fastText and transformer) based models to set the ground for future studies. The dataset and associated resources are publicly available at https://crisisnlp.qcri.org/humaid_dataset.html.

Downloads

Published

2021-05-22

How to Cite

Alam, F., Qazi, U., Imran, M., & Ofli, F. (2021). HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks. Proceedings of the International AAAI Conference on Web and Social Media, 15(1), 933-942. https://doi.org/10.1609/icwsm.v15i1.18116