Some Like It Small: Czech Semantic Embedding Models for Industry Applications


  • Jiří Bednář
  • Jakub Náplava
  • Petra Barančíková
  • Ondřej Lisický



Search, Deep Learning and Neural Networks , Machine Learning, Natural Language , Track: Deployed Applications


This article focuses on the development and evaluation of Small-sized Czech sentence embedding models. Small models are important components for real-time industry applications in resource-constrained environments. Given the limited availability of labeled Czech data, alternative approaches, including pre-training, knowledge distillation, and unsupervised contrastive fine-tuning, are investigated. Comprehensive intrinsic and extrinsic analyses are conducted, showcasing the competitive performance of our models compared to significantly larger counterparts, with approximately 8 times smaller size and 5 times faster speed than conventional Base-sized models. To promote cooperation and reproducibility, both the models and the evaluation pipeline are made publicly accessible. Ultimately, this article presents practical applications of the developed sentence embedding models in, the Czech search engine. These models have effectively replaced previous counterparts, enhancing the overall search experience for instance, in organic search, featured snippets, and image search. This transition has yielded improved performance.



How to Cite

Bednář, J., Náplava, J., Barančíková, P., & Lisický, O. (2024). Some Like It Small: Czech Semantic Embedding Models for Industry Applications. Proceedings of the AAAI Conference on Artificial Intelligence, 38(21), 22734-22742.



IAAI Technical Track on Deployed Highly Innovative Applications of AI