TY - JOUR AU - Ni, Lin AU - Bao, Qiming AU - Li, Xiaoxuan AU - Qi, Qianqian AU - Denny, Paul AU - Warren, Jim AU - Witbrock, Michael AU - Liu, Jiamou PY - 2022/06/28 Y2 - 2024/03/28 TI - DeepQR: Neural-Based Quality Ratings for Learnersourced Multiple-Choice Questions JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 36 IS - 11 SE - EAAI Symposium: Full Papers DO - 10.1609/aaai.v36i11.21562 UR - https://ojs.aaai.org/index.php/AAAI/article/view/21562 SP - 12826-12834 AB - Automated question quality rating (AQQR) aims to evaluate question quality through computational means, thereby addressing emerging challenges in online learnersourced question repositories. Existing methods for AQQR rely solely on explicitly-defined criteria such as readability and word count, while not fully utilising the power of state-of-the-art deep-learning techniques. We propose DeepQR, a novel neural-network model for AQQR that is trained using multiple-choice-question (MCQ) datasets collected from PeerWise, a widely-used learnersourcing platform. Along with designing DeepQR, we investigate models based on explicitly-defined features, or semantic features, or both. We also introduce a self-attention mechanism to capture semantic correlations between MCQ components, and a contrastive-learning approach to acquire question representations using quality ratings. Extensive experiments on datasets collected from eight university-level courses illustrate that DeepQR has superior performance over six comparative models. ER -