Saudi Cultural Missions Theses & Dissertations
Permanent URI for this communityhttps://drepo.sdl.edu.sa/handle/20.500.14154/10
Browse
4 results
Search Results
Item Restricted Deep Reinforcement Learning for Real-Time Energy Management in Community Microgrids(Lancaster University, 2025) Aldahmashi, Jamal; Ma, XiandongThe integration of renewable energy sources (RESs), energy storage systems (ESSs), and the electrification of transportation are driving a rapid transformation of modern power systems. These changes not only provide great potential to reduce carbon emissions, increase sustainability and improve reliability, but also present complex challenges. Traditional and centralized power systems were originally designed for one-way power flow, from large power plants to consumers, and are becoming increasingly inadequate in the face of intermittent renewable generation, distributed and variable loads, and heightened risks of severe weather disturbances. Thus, intelligent, adaptive and resilient methods for energy management have become a critical priority. In this thesis, I address the need for advanced, real-time control in modern power systems through the use of deep reinforcement learning (DRL) to optimize active and reactive power flows under uncertainty. First, a model-free framework for a single home energy management system (HEMS) that integrates photovoltaic (PV) panels, ESSs, electric vehicles (EVs), and multiple types of residential loads. In contrast to existing methods that focus on active power flows alone, the proposed method optimizes reactive power to improve power factor and avoid possible financial penalties. This framework adapts to fluctuating renewable generation, uncertain EV charging profiles, and the unpredictable behavior of loads by using DRL algorithms that can learn directly from interactions with the environment without explicit mathematical models. Real world data tests show over 30% electricity cost savings and substantial power factor improvements. To extend this concept from individual homes to larger communities, a community energy management system (CEMS) is proposed. Multiple smart homes, each equipped with a HEMS, are interconnected through a point of common coupling to form a community microgrid (CMG). Each home acts as a local agent making autonomous decisions, and a multi-agent DRL (MADRL) architecture is employed to coordinate their actions in a decentralized yet cooperative manner. Further, electricity price forecasting is integrated with a Long Short Term Memory (LSTM) network for proactive scheduling of flexible loads. Simulation results show that this data driven, cooperative control approach can reduce overall community electricity costs by up to 29.66% and keep community voltages more stable than conventional centralized and model-based methods. Also, the proposed MADRL strategy retains decision making at the household level, which provides benefits in terms of privacy, scalability, and adaptability to various grid conditions. The thesis then incorporates optimal power flow (OPF) constraints into the energy management system (EMS) for CMG with high penetration of renewables, ESSs and EVs, recognizing that even larger scale distribution networks require advanced coordination. The work reformulates the OPF problem as a Markov decision process (MDP) and uses a dual-layer DRL structure. The objectives of the first layer controls for continuous control of active power using a twin delayed deep deterministic policy gradient (TD3) algorithm with cost minimization, load shedding prevention and efficient use of DERs. The second layer, which uses a double deep Q-network (DDQN), controls discrete reactive power to maintain voltage stability. This dual-layer approach addresses the challenges of high-dimensional, non-linear, and stochastic power systems. The experiments on a modified IEEE-15 bus system demonstrate up to 10.41% cost savings versus no EMS, with less voltage violations and less load shedding. The dual-layer DRL framework is resilient to stochastic variations in renewable output and load demand, and is a practical candidate for real-time distribution network operations. Overall, the research presented demonstrates that DRL-based solutions, whether applied to individual homes, local communities or larger distribution networks, can successfully deal with the uncertainty and variability of modern power systems. By integrating cutting-edge neural network architectures for price forecasting, multi-agent coordination, and dual-layer control, the proposed methods outperform traditional optimization and control approaches in terms of cost efficiency, voltage stability, and scalability. As a result, these techniques offer great potential for enabling flexible, economically viable, and robust power grid operations. With increasing proportion of RESs, ESSs and EVs, the demand for such intelligent, adaptive and decentralized energy management solutions will increase, leading to a more sustainable and resilient electricity infrastructure.16 0Item Restricted DEEP LEARNING APPROACH FOR EFFICIENT ENERGY CONSUMPTION AND HIGH THROUGHPUT IN MOBILE WIRELESS SENSOR NETWORKS(Lancaster University, 2024-08-24) Alsalmi, Nada; Navaie, Keivan; Rahmani, HosseinFeatures such as scalability, smaller size, simplicity, low-cost operation, selforganization abilities, and easy and fast deployment are the main parameters of a Wireless Sensor Network (WSN). The research demand is growing on WSNs, and therefore, areas under agriculture, industry, healthcare, manufacturing, security, surveillance, transport, air quality, water quality, etc., have started to possess the attributes of WSNs. The primary goal of SNs is to collect the data from the area of interest and communicate it to the sink or base station (BS) for further processing via single or multi-hop transmission. Sometimes, the BS acts as a gateway to the Internet of Things (IoT), where the IoT can communicate the data to the Cloud using the Internet. The battery-equipped SNs consume more energy for heavy data transmission. Transmission of high-quality data in SN makes the battery-equipped micro-sensors consume much energy. Mobile Wireless Sensor Network (MWSN) represents a fast-evolving technology, and its use in many things is not limited. While fixed-infrastructure networks constrain sensor nodes to one specific location, MWSNs allow the partial nodes or all nodes to move wherever they want and communicate between themselves, making the whole system more flexible. Furthermore, MWSNs can be compared with respect to GPS, Bluetooth Low Energy (BLE), and existing wireless sensor networks in aspects of extended network lifespan, energy saving, multiband functionality, and high targeting. Nevertheless, pathfinding in MWSNs is very challenging since the sensor nodes are mobile, low-cost devices that are timeconstrained, allowing limited resources to be used. On the mobile network, this unique frequency scheme creates extra difficulty in routing. In most monitoring applications, only partial nodes need to be moved in the network. Such nodes are called mobile agent sink nodes or sensor nodes. In the present work, the movement of only a few nodes is considered in MWSN. Energy consumption and network connectivity are two major issues in MWSNs. Several studies have been conducted to develop and propose suitable solutions for these problems. Many researchers are working to develop the best solutions due to the severe problems with energy consumption and network connectivity in mobile wireless sensor networks. To investigate network connectivity, this study introduces a new efficient technique that considers parameters like network stability, detection area, low energy consumption, etc. This approach guarantees network connectivity, communication sustainability, and the highest level of energy consumption optimization. This research investigates network connectivity issue and proposes two routing algorithms, namely Self-Organizing Maps based-Optimized Link State Routing (SOM-OSLR) and Deep Reinforcement Learning based-Optimized Link State Routing (DRL-OLSR) for MWSNs. Both algorithms undertake the relationship between sensor node deployment, communication radius, and detection area and suggest a new way to maintain communication while optimizing energy usage. I have evaluated both algorithms through simulations by considering various performance metrics such as connection probability, end-to-end delay, overhead, network throughput, and energy consumption. The network is analyzed for proposed routing and aggregation methods to analyze the performance. The simulation analysis is discussed under three scenarios. The first scenario undertakes ’no optimization,’ the second considers SOM-OLSR, and the third undertakes DRL-OLSR. The simulation results indicate that the SOM-OLSR performs better compared to the case with ’no routing’ optimization. Comparing DRL-OLSR and SOM-OLSR indicates that the former outperforms the latter in terms of low latency and high network lifetime. Specifically, the DRL-OLSR achieves a 47% higher throughput and 67% lower energy consumption compared to the SOM-OLSR. In addition, when compared to the ’No optimization’ condition, the DRL-OLSR achieves a notable 69.7% higher throughput and almost 89% lower energy consumption. These findings highlight the effectiveness of the DRL-OLSR approach in optimizing network performance and energy efficiency in wireless sensor networks. Similarly, data aggregation consistently reduces energy consumption across all scenarios, with up to 50% lower as compared to without data aggregation.32 0Item Restricted Deep Reinforcement Learning and Privacy Preserving with Differential Privacy(Saudi Digital Library, 2023-09-15) Abahussein, Suleiman; Zhu, TianqingWith the rapid advances in technology in the current era and the emergence of multiple technologies that have transformed society, Deep reinforcement learning (DRL) offers promising solutions and enhanced capabilities with demonstrated superior results. Deep reinforcement learning is a subfield of artificial intelligence that has attracted significant research attention and development over the past few years. Reinforcement learning (RL) is enabled by deep learning to address the intractable problems previously encountered, for example, how an agent learns to play video games with only pixels as input. In the field of robotics, deep reinforcement learning algorithms are employed where control policies for robots can be learned directly from camera inputs in the real world. Deep RL aims to maximize the cumulative reward through the process of trial and error to find the optimal policy. The learning method is carried out by executing the action, receiving corresponding rewards and then moving to the next state. In some complex problems, it is necessary to have more than one RL agent, which leads to the idea of multi-agent reinforcement learning, where more than one agent works together and shares the same environment to achieve a certain goal. An example of multi-agent RL is multiple robotics working to rescue an individual. Nowadays, deep reinforcement learning is used in various areas, such as recommendation systems, robotics, and health applications. While there are enormous benefits to using these technologies, there are also significant privacy concerns associated with them. The learning process in deep reinforcement learning involves performing the action, receiving the reward, and moving to the next state. Deep reinforcement learning is vulnerable to adversary attacks, and private information can be inferred by an adversary using recursive querying. The trained policy could be released to the client side, which could enable the adversary to infer private information from the trained policy, pose a real risk, and constitute a breach of privacy. This research focuses on deep reinforcement learning and multi-agent reinforcement learning and the related privacy issues. The contributions made by this research are as follows: • This research proposes a solution for online food delivery services to increase the number of food delivery orders and thereby increase the long-term income of couriers. The solution involves leveraging multi-agent reinforcement learning by employing two multi-agent reinforcement learning algorithms to guide couriers to areas with a high demand for food delivery orders. • This research proposes a solution to protect privacy in Double and Dueling Deep Q Networks by adopting the Differentially Private Stochastic Gradient Descent (DPSGD) method and injecting Gaussian noise into the gradient. • This research proposes the Protect User Location Method (PULM) to protect customer location information in online food delivery services. This method injects differential privacy Laplace noise based on two factors: the size of the city and the frequency of customer orders. • This research proposes a Protect Trajectory and Location in Food Delivery (PTLFD) method to maintain the privacy of the customer’s stored data in online food delivery services. This method leverages multi-agent reinforcement learning and differential privacy to protect customer location information.27 0Item Restricted Applications Of Artificial Intelligence In Supply Chain Management In The Era Of Industry 4.0(2023) Ali, Arishi; Krishna, KrishnanNowadays, an emerging trend in Supply Chain Management (SCM) is a focus shift from classical Supply Chain (SC) to digital SC. However, decisions in the digital SC context require new tools and methodologies that consider the digitalization environment. Artificial Intelligence (AI) methodologies can provide learning, predictive, and automated decision-making capabilities in the digital environment. Among a wide range of problems in the SCM field, risk management, logistics, and transportation have received less attention from an AI perspective. The work presented in this dissertation proposes three AI-based approaches to help SCs manage their operations more effectively using creative risk monitoring and logistics/transportation solutions in the era of Industry 4.0. In the first study, a Digital Twin (DT) framework for analyzing and predicting the impact of COVID-19 disruptions on the manufacturing SC is developed to support the decision-making process in disrupted SC. The proposed Digital SC Twin (DSCT) model is aimed to work as an online controlling tower to monitor the behavior of physical SC in the digital environment and guide SCM managers to make the necessary adjustments to minimize risks and maintain SC stability during disruptions. In the second study, a contactless truck-drone delivery model for last-mile problems in the SC is introduced to support logistics and transportation operations during pandemics. A hybrid AI approach is developed to provide quality real-time solutions for the introduced truck-drone delivery system. In the third study, a collaborative Multi-Agent Deep Reinforcement Learning (MADRL) approach for vehicle routing in the SCM is designed to facilitate collaboration and communication among multiple vehicles in the SC distribution networks. Overall, the methods and models presented in this dissertation can enable SCs to transform their traditional practices, provide cost savings, support real-time decision-making, and enable self-optimization and self-healing capabilities in the age of Industry 4.056 0