Instruction-tuned generative AI for disability access: a comparative evaluation using cosine similarity


KILINÇ M., ALTINTAŞ V.

Knowledge and Information Systems, 2025 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Basım Tarihi: 2025
  • Doi Numarası: 10.1007/s10115-025-02585-1
  • Dergi Adı: Knowledge and Information Systems
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, PASCAL, ABI/INFORM, Applied Science & Technology Source, Compendex, Computer & Applied Sciences, INSPEC
  • Anahtar Kelimeler: Accessible AI, Cosine similarity, GPT for accessibility, Instruction tuning
  • Manisa Celal Bayar Üniversitesi Adresli: Evet

Özet

Ensuring equitable access to information, communication, and education for individuals with disabilities remains a critical challenge in modern societies. Existing AI technologies, particularly general-purpose large language models (LLMs), often fall short in addressing the specific needs of people with disabilities, leading to digital exclusion and limited participation. In response, this study introduces ApAI, an instruction-tuned generative GPT model tailored to enhance accessibility for individuals with disabilities. ApAI was trained on regulatory documents and real-world scenarios sourced from disability-focused institutions in Türkiye, allowing the model to adapt to domain-specific contexts through instruction tuning. The effectiveness of ApAI was evaluated using the cosine similarity method and benchmarked against leading LLMs ChatGPT 4.0, Claude 3.5 Sonnet, and Gemini across four accessibility-focused prompts. ApAI consistently achieved the highest semantic alignment, with cosine similarity scores of 0.9291, 0.7334, 0.9598, and 0.9441, respectively. Notably, ApAI outperformed the next best model by 2.9% in Prompt-1, 6.6% in Prompt-2, and 1.4% in Prompt-3, demonstrating its superior contextual relevance and instruction-following capability. While Claude 3.5 Sonnet slightly outperformed ApAI in Prompt-4, ApAI showed overall stronger alignment across diverse scenarios. These results highlight the value of instruction tuning in building inclusive AI systems and underline the potential of customized generative models in addressing accessibility challenges. With its domain-specific adaptability, strong semantic accuracy, and ethical foundation, ApAI represents a promising step toward inclusive AI design. The study emphasizes the importance of future research focused on expanding datasets, validating prompt design with user feedback, and integrating multimodal capabilities for broader accessibility impact.