MODELING AND REASONING WITH NON-FUNCTIONAL REQUIREMENTS USING GENERATIVE AI

Abstract

Non-functional requirements (NFRs), such as usability and security, can often be subjective, and achieving one NFR can help or hurt other NFRs, as different people may view, interpret, and evaluate them differently. This challenge is compounded because NFRs are often stated briefly and vaguely in informal, natural language. However, this informal NFR description makes it difficult to detect deficiencies, reason about trade-offs, and model NFRs correctly using Softgoal Interdependency Graphs (SIGs). As a result, the practice of NFR modeling remains limited, partly due to unfamiliarity with modeling languages like SIG and insufficient understanding of relevant NFRs. Recently, Generative AI (GenAI) demonstrated some familiarity with NFRs and SIG modeling concepts, such as goal decomposition and operationalization; however, it often lacks formal syntax of SIG-specific syntax, which limits its ability to generate correct NFR models. Training GenAI on SIG modeling is also not an easy task, as there are few real-world examples of SIG models. This dissertation presents the ReGenAI framework for generating and translating Informal NFR descriptions to Semi-formal SIG Models. We propose five main technical contributions. First, a domain-independent, activity-oriented ontology and process for ReGenAI are explicitly and formally presented to describe categories of essential SIG concepts, relationships, and constraints needed to generate and transform informal NFR descriptions into semi-formal SIG models. The ontology and process ensure traceability from textual statements to SIG model elements while reducing omissions and commissions in the resulting model. Second, a Backus–Naur Form (BNF)-based formal textual grammar was developed to enforce syntactic correctness and constrain GenAI-based generation, guiding the GenAI to generate SIGs that align with formal notation. Third, the SIG-GPT method is introduced as a GenAI-based generator grounded in retrieval-augmented generation (RAG) and constrained by textual grammar, enabling the generation and translation of SIG structures that align with SIG formal notation and ensuring it is ready for seamless integration with visual modeling tools. Fourth, a set of formalized validation rules is described for semantic reasoning to identify and detect modeling gaps and inconsistencies in the generated SIG models, ensuring that the models preserve the intended meaning defined in the SIG ontology. Fifth, a repair method is presented to complete and correct the detected deficiencies by repairing the missing parts into a final validated SIG model using retrieval-augmented generation (RAG) grounded in external NFR knowledge sources. To see the strengths and weaknesses of the ReGenAI framework, two experimental studies were conducted using PURE, a dataset of public requirements documents, and FISMA, a U.S. federal information security regulation, as a realistic case studies to produce semi-formal SIG models from informal NFR descriptions and to detect modeling deficiencies in transforming the source descriptions to the target models. We believe that our proposed framework can help generate, validate, and repair modeling deficiencies that negatively affect an NFR goal, providing insights into the detected gaps and how they impact the generated SIG model.