Evaluating Task-Dependent Taxonomies for Navigation

Authors

  • Yuyin Sun University of Washington
  • Adish Singla ETH Zurich
  • Tori Yan University of Washington
  • Andreas Krause EHT Zurich
  • Dieter Fox University of Washington

DOI:

https://doi.org/10.1609/hcomp.v4i1.13286

Keywords:

Taxonomy, Information retrieval

Abstract

Taxonomies of concepts are important across many application domains, for instance, online shopping portals use catalogs to help users navigate and search for products. Task-dependent taxonomies, e.g., adapting the taxonomy to a specific cohort of users, can greatly improve the effectiveness of navigation and search. However, taxonomies are usually created by domain experts and hence designing task-dependent taxonomies can be an expensive process: this often limits the applications to deploy generic taxonomies. Crowdsourcing-based techniques have the potential to provide a cost-efficient solution to building task-dependent taxonomies. In this paper, we present the first quantitative study to evaluate the effectiveness of these crowdsourcing based techniques. Our experimental study compares different task-dependent taxonomies built via crowdsourcing and generic taxonomies built by experts. We design randomized behavioral experiments on the Amazon Mechanical Turk platform for navigation tasks using these taxonomies resembling real-world applications such as product search. We record various metrics such as the time of navigation, the number of clicks performed, and the search path taken by a participant to navigate the taxonomy to locate a desired object. Our findings show that task-dependent taxonomies built by crowdsourcing techniques can reduce the navigation time up to $20\%$. Our results, in turn,demonstrate the power of crowdsourcing for learning complex structures such as semantic taxonomies.

Downloads

Published

2016-09-21

How to Cite

Sun, Y., Singla, A., Yan, T., Krause, A., & Fox, D. (2016). Evaluating Task-Dependent Taxonomies for Navigation. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 4(1), 229-238. https://doi.org/10.1609/hcomp.v4i1.13286