Saudi Cultural Missions Theses & Dissertations
Permanent URI for this communityhttps://drepo.sdl.edu.sa/handle/20.500.14154/10
Browse
3 results
Search Results
Item Restricted Fine-Tuning Large Language Models: A Systematic Review of Methods, Challenges, and Domain- Specific Adaptations(Saudi Digital Library, 2025) Alharbi, Shaima; Hussain, FarookhFine-tuning large language models (LLMs) has emerged as a crucial step for adapting these powerful models to specialized tasks and domains. In this paper, we present a systematic literature review of recent techniques for fine-tuning LLMs, the challenges encountered across different application domains, and the strategies developed to address domain- specific requirements. We identify four key requirements for effective fine-tuning: (1) Parameter-efficient and scalable methods that mitigate the resource cost of updating billion- parameter models, (2) High-quality, low-cost data usage techniques for curating or generating training data, (3) Domain adaptability and knowledge integration approaches (including retrieval augmentation and alignment with knowledge bases), and (4) Robust evaluation and interpretability practices to ensure fine-tuned models are accurate and trustworthy. We analyze six representative papers in diverse domains – healthcare (biomedical LLMs like Med-PaLM and BioGPT), recommender systems (e.g. the DEALRec data-efficient tuning framework), smart manufacturing (knowledge-graph- augmented RAG pipelines), socially-informed AI (instruction- tuned models like FLAN and LLaMA-based alignments), and education (comparing specialized small models to GPT-4 with retrieval). Through this analysis, we synthesize how each approach fulfills or falls short of the identified requirements. Our review highlights emerging trends such as parameter- efficient fine-tuning (PEFT), retrieval augmented generation (RAG), and multi-task instruction tuning as promising directions to specialize LLMs while controlling cost and maintaining performance. We discuss open challenges including the trade-off between efficiency and performance, data bias and scarcity, maintaining generalization across domains, and improving evaluation metrics and interpretability. Finally, we outline future research opportunities to further enhance the fine-tuning of LLMs for domain-specific applications.23 0Item Restricted Fine-Tuning Large Language Models A Systematic Review of Methods Challenges and Domain-Specific Adaptations(Saudi Digital Library, 2025) Altalhi, Sarah; Albaqami, Norah; Alharbi, Shaima; Hussain, FarookhFine-tuning large language models (LLMs) has emerged as a crucial step for adapting these powerful models to specialized tasks and domains. In this paper, we present a systematic literature review of recent techniques for fine-tuning LLMs, the challenges encountered across different application domains, and the strategies developed to address domain-specific requirements. We identify four key requirements for effective fine-tuning: (1) Parameter-efficient and scalable methods that mitigate the resource cost of updating billion-parameter models, (2) High-quality, low-cost data usage techniques for curating or generating training data, (3) Domain adaptability and knowledge integration approaches (including retrieval augmentation and alignment with knowledge bases), and (4) Robust evaluation and interpretability practices to ensure fine-tuned models are accurate and trustworthy. We analyze six representative papers in diverse domains – healthcare (biomedical LLMs like Med-PaLM and BioGPT), recommender systems (e.g. the DEALRec data-efficient tuning framework), smart manufacturing (knowledge-graph-augmented RAG pipelines), socially-informed AI (instruction-tuned models like FLAN and LLaMA-based alignments), and education (comparing specialized small models to GPT-4 with retrieval). Through this analysis, we synthesize how each approach fulfills or falls short of the identified requirements. Our review highlights emerging trends such as parameter-efficient fine-tuning (PEFT), retrieval augmented generation (RAG), and multi-task instruction tuning as promising directions to specialize LLMs while controlling cost and maintaining performance. We discuss open challenges including the trade-off between efficiency and performance, data bias and scarcity, maintaining generalization across domains, and improving evaluation metrics and interpretability. Finally, we outline future research opportunities to further enhance the fine-tuning of LLMs for domain-specific applications.11 0Item Restricted MULTIDIMENSIONAL APPROACHES IN BUG DETECTION FOR PARALLEL PROGRAMMING AND TEXT-TO-CODE SEMANTIC PARSING(University of Central Florida, 2025) Alsofyani, May; Wang LiqiangThis dissertation applies deep learning and large language models to two domains: parallel programming fault detection and text-to-code translation, aiming to enhance software reliability and natural language-driven code generation. Due to their unpredictable nature, concurrency bugs-particularly data race bugs— present significant challenges in fault detection for parallel programming. We investigate deep learning and LLM-based approaches for detecting data race bugs in OpenMP programs. Our proposed methods include a transformer encoder and GPT-4 through prompt engineering and fine-tuning. Experimental results demonstrate that the transformer encoder achieves competitive accuracy compared to LLMs, highlighting its effectiveness in understanding complex OpenMP directives. Expanding this research, we explore the role of LLMs in detecting faults in Pthreads, which requires a deep understanding of thread-based logic and synchronization mechanisms. We analyze ChatGPT's effectiveness in Pthreads fault detection through dialogue-based interactions and advanced prompt engineering techniques, including Zero-Shot, Few-Shot, Chain-of-Thought, and Retrieval-Augmented Generation. Additionally, we introduce three hybrid prompting techniques—Chain-of-Thought with Few-Shot Prompting, Retrieval-Augmented Generation with Few-Shot Prompting, and Prompt Chaining with Few-Shot Prompting—to enhance fault detection performance. In the semantic parsing domain, our research bridges the gap between natural language and executable code, focusing on text-to-SQL translation. To address SQL's limitations in statistical analysis, we introduce SIGMA, a dataset for text-to-code semantic parsing with statistical analysis capabilities. In addition, we address the gap in cross-domain context-dependent text-to-SQL translation for the Arabic language. While prior research has focused on English and Chinese datasets, no efforts have been made to explore Arabic cross-domain conversational querying. We introduce Ar-SParC, the first Arabic cross-domain, context-dependent text-to-SQL dataset. This dissertation contributes to fault detection in parallel programming and semantic parsing with statistical analysis, leveraging cutting-edge deep learning and LLMs techniques. Our findings advance bug detection in high-performance computing and natural language-based code generation, significantly improving software reliability and accessibility.30 0