LITMUS Predictor: An AI Assistant for Building Reliable, High-Performing and Fair Multilingual NLP Systems
Keywords:Multilingual Language Models, Performance Prediction, Crosslingual Zeroshot Transfer, LITMUS, Data Labeling
AbstractPre-trained multilingual language models are gaining popularity due to their cross-lingual zero-shot transfer ability, but these models do not perform equally well in all languages. Evaluating task-specific performance of a model in a large number of languages is often a challenge due to lack of labeled data, as is targeting improvements in low performing languages through few-shot learning. We present a tool - LITMUS Predictor - that can make reliable performance projections for a fine-tuned task-specific model in a set of languages without test and training data, and help strategize data labeling efforts to optimize performance and fairness objectives.
How to Cite
Srinivasan, A., Kholkar, G., Kejriwal, R., Ganu, T., Dandapat, S., Sitaram, S., Santhanam, B., Aditya, S., Bali, K., & Choudhury, M. (2022). LITMUS Predictor: An AI Assistant for Building Reliable, High-Performing and Fair Multilingual NLP Systems. Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), 13227-13229. https://doi.org/10.1609/aaai.v36i11.21736
AAAI Demonstration Track