LITMUS Predictor: An AI Assistant for Building Reliable, High-Performing and Fair Multilingual NLP Systems
DOI:
https://doi.org/10.1609/aaai.v36i11.21736Keywords:
Multilingual Language Models, Performance Prediction, Crosslingual Zeroshot Transfer, LITMUS, Data LabelingAbstract
Pre-trained multilingual language models are gaining popularity due to their cross-lingual zero-shot transfer ability, but these models do not perform equally well in all languages. Evaluating task-specific performance of a model in a large number of languages is often a challenge due to lack of labeled data, as is targeting improvements in low performing languages through few-shot learning. We present a tool - LITMUS Predictor - that can make reliable performance projections for a fine-tuned task-specific model in a set of languages without test and training data, and help strategize data labeling efforts to optimize performance and fairness objectives.Downloads
Published
2022-06-28
How to Cite
Srinivasan, A., Kholkar, G., Kejriwal, R., Ganu, T., Dandapat, S., Sitaram, S., Santhanam, B., Aditya, S., Bali, K., & Choudhury, M. (2022). LITMUS Predictor: An AI Assistant for Building Reliable, High-Performing and Fair Multilingual NLP Systems. Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), 13227-13229. https://doi.org/10.1609/aaai.v36i11.21736
Issue
Section
AAAI Demonstration Track