Developing an Advanced Software Requirements Classification Model Using BERT: An Empirical Evaluation Study on Newly Generated Turkish Data

YÜCALAR F.

Applied Sciences (Switzerland), vol.13, no.20, 2023 (SCI-Expanded, Scopus) identifier identifier

  • Publication Type: Article / Article
  • Volume: 13 Issue: 20
  • Publication Date: 2023
  • Doi Number: 10.3390/app132011127
  • Journal Name: Applied Sciences (Switzerland)
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, Agricultural & Environmental Science Database, Applied Science & Technology Source, Communication Abstracts, INSPEC, Metadex, Directory of Open Access Journals, Civil Engineering Abstracts
  • Keywords: software requirements classification, transformer learning, deep neural networks, machine learning, functional requirements, non-functional requirements
  • Manisa Celal Bayar University Affiliated: Yes

Abstract

Requirements Engineering (RE) is an important step in the whole software development lifecycle. The problem in RE is to determine the class of the software requirements as functional (FR) and non-functional (NFR). Proper and early identification of these requirements is vital for the entire development cycle. On the other hand, manual identification of these classes is a timewaster, and it needs to be automated. Methodically, machine learning (ML) approaches are applied to address this problem. In this study, twenty ML algorithms, such as Naïve Bayes, Rotation Forests, Convolutional Neural Networks, and transformers such as BERT, were used to predict FR and NFR. Any ML algorithm requires a dataset for training. For this goal, we generated a unique Turkish dataset having collected the requirements from real-world software projects with 4600 samples. The generated Turkish dataset was used to assess the performance of the three groups of ML algorithms in terms of F-score and related statistical metrics. In particular, out of 20 ML algorithms, BERTurk was found to be the most successful algorithm for discriminating FR and NFR in terms of a 95% F-score metric. From the FR and NFR identification problem point of view, transformer algorithms show significantly better performances.