Machine Learning Ensemble Methods for Classifying Multi-media Data

Abstract

Multimedia data have, over recent years, been produced in many elds. They have important applications for such diverse areas as social media and healthcare, due to their capacity to capture rich information. However, their unstructured and separated nature gives rise to various problems. In particular, fusing and integrating multi-media datasets and nding eective ways to learn from them have proven to be major challenges for machine learning. In this thesis we investigated the development of the ensemble methods for classifying multi-media data in two key aspects: data fusion and model selection. For the data fusion, we devised two dierent strategies. The rst one is the Feature Level Ensemble Method (FLEM) that aggregates all the features into a single dataset and then generates the models to build ensembles using this dataset. The second one is the Decision Level Ensemble Method (DLEM) that generates the models from each sub dataset individually and then aggregates their outputs with a decision fusion function. For the model selection we derived four dierent model selection rules. The rst rule, R0, uses just the accuracy to select models. The rules R1 and R2 use rstly accuracy and then diversity to select models. In R3, we dened a generalised function that combines the accuracy and diversity with dierent weights to select models to build an ensemble. Our methods were compared with existing well known ensemble methods using the same dataset and another dataset that became available after our methods had been developed. The results were critically analysed and the statistical signi cance analyses of the results show that our methods had better performance in general and the generalised R3 is the most eective rule in building ensembles.