SafeGenChat – a Neuro-Symbolic Approach to Dialogs for Trustworthy Information Retrieval Conversations on Sensitive Topics

Abstract

Large Language Model (LLM)-based conversational assistants, such as ChatGPT, Gemini, and DeepSeek, have shown strong potential for enabling natural language access to information. However, their deployment in sensitive domains raises concerns related to trustworthiness, including hallucinations, limited transparency, and the risk of unsafe or inappropriate responses. Purely rule-based conversational systems, while reliable and accurate to the information source, lack the flexibility required for open-ended information retrieval. We introduce SafeGenChat, a neuro-symbolic hybrid framework for trustworthy information retrieval dialogs on sensitive topics where it is paramount to make the context and risk of the information transparent to the user. Inspired by the dual-system theory of fast and slow thinking as implemented in the recently proposed SOFAI architecture, SafeGenChat combines a generative LLM-based component (System-1) with a symbolic, rule-based component (System-2) that dynamically routes user queries between verified answers and purposeful do-not-answer responses based on an assessed risk of the dialog context. We present a case study of an HIV-focused chatbot that answers user queries related to HIV to illustrate the design and application of SafeGenChat in a safety-critical domain. Overall, this work introduces a neuro-symbolic framework for risk-aware conversational information retrieval, adapts the SOFAI dual-system architecture to dialog-based settings, and demonstrates its applicability through a safety-critical HIV decision support case study.