Akademik Veri Yönetim Sistemi
Turkish Journal of Engineering, cilt.9, sa.3, ss.544-559, 2025 (Scopus)
Religious teachings can sometimes be complex and challenging to grasp,
but chatbots can serve as effective assistants in this domain. Large
Language Model (LLM) based chatbots, powered by Natural Language
Processing (NLP), can connect related topics and provide well-supported
responses to intricate questions, making them valuable tools for
religious education. However, LLMs are prone to hallucinations as they
can generate inaccurate or irrelevant information, and these can include
sensitive content that could be offensive, inappropriate, or
controversial. Addressing such topics without inadvertently promoting
hate speech or disrespecting certain beliefs remains a significant
challenge. As a solution to these issues, we introduce MufassirQAS, a
system that enhances LLM accuracy and transparency using a vector
database-driven Retrieval-Augmented Generation (RAG) approach. We built a
dataset comprising fundamental books containing Turkish translations
and interpretations of Islamic texts. This database is leveraged to
answer religious inquiries while ensuring that responses remain reliable
and contextually grounded. Our system also presents the relevant
dataset sections alongside the LLM-generated answers, reinforcing
transparency. We carefully designed system prompts to prevent harmful,
offensive, or disrespectful outputs, ensuring that responses align with
ethical and respectful discourse. Moreover, MufassirQAS provides
supplementary details, such as source page numbers and referenced
articles, to enhance credibility. To evaluate its effectiveness, we
tested MufassirQAS against ChatGPT with sensitive questions, and our
system demonstrated superior performance in maintaining accuracy and
reliability. Future work will focus on improving accuracy and refining
prompt engineering techniques to further minimize biases and ensure even
more reliable responses.