Activation Functions In Deep Learning For Aerial Image Segmentation

Abstract

In remote sensing, deep learning models have been widely proposed and evaluated, especially for scene classification using Convolutional Neural Networks (CNNs) or semantic segmentation through Fully Convolutional Networks (FCN). There is still a research gap in studying the impact of activation functions on semantic segmentation performance in FCN, mainly when applied to aerial images. This dissertation attempts to bridge this gap by comprehensively examining the impact of nine activation func- tions on FCN models. This study presents intensive experiments on different FCN architectures, UNet and FPN. UNet is a simple and straightforward architecture, while FPN is very deep and complex. Also, two datasets were used: a small dataset with only five classes with images from the same country and a more diverse dataset with nine classes and images of various resolutions and complexity from all over the world. This experiment consists of two phases. The first phase involves establishing four baseline models for integrating diverse activation functions through a systematic method of hy- perparameter tuning. Afterwards, each baseline model was implemented across ten different activation function variations. In total, forty distinct models were trained and evaluated. Based on these experiments, it is evident that the choice of activation func- tions has a significant impact on the stability of the training and convergence speed. Additionally, the activation functions play a crucial role in the overall performance and within-class performance of the models. However, the behaviour of each activa- tion function is highly affected by the combination of architectures and datasets used.