One Word, One Command, One Translation: Combining GenAI and Traditional Approaches in Simulating In-Situ Spoken Human Interaction for Medical and Engineering Applications

Abstract

The proposed deployment of knowledge graphs bridges neural networks and other forms of data-driven processing with traditional strategies such as template-based, slot-filling frameworks and controlled-language like restrictions. The present approach is based on knowledge graphs enabling the role of context-specific dimensions of singular words uttered in HCI applications. The cases concern medical and engineering applications and speaker-related / environmental factors disrupting communication. Paralinguistic features such as deictic gestures and other types of implied information not uttered / not correctly processed by Speech Recognition constitute an additional parameter in the proposed approach.