Crowdsourced Testing Approach For Mobile Compatibility Testing

Abstract

The frequent release of mobile devices and operating system versions bring several compatibility issues to mobile applications. This thesis addresses fragmentation-induced compatibility issues. The thesis comprises three main phases. The first of these involves an in-depth review of relevant literature that identifies the main challenges of existing compatibility testing approaches. The second phase reflects on the conduction of an in-depth exploratory study on Android/iOS developers in academia and industry to gain further insight into their actual needs in testing environments whilst gauging their willingness to work with public testers with varied experience. The third phase relates to implementing a new manual crowdtesting approach that supports large-scale distribution of tests and execution by public testers and real users on a larger number of devices in a short time. The approach is designed based on a direct crowdtesting workflow to bridge the communication gap between developers and testers. The approach supports performing the three dimensions of compatibility testing. This approach helps explore different behaviours of the app and the users of the app to identify all compatibility issues. Two empirical evaluation studies were conducted on iOS/Android developers and testers to gauge developers’ and testers’ perspectives regarding the benefits, satisfaction, and effectiveness of the proposed approach. Our findings show that the approach is effective and improves on current state-of-the-art approaches. The findings also show that the approach met the several unmet needs of different groups of developers and testers. The evaluation proved that the different groups of developers and testers were satisfied with the approach. Importantly, the level of satisfaction was especially high in small and medium-sized enterprises that have limited access to traditional testing infrastructures, which are instead present in large enterprises. This is the first research that provides insights for future research into the actual needs of each group of developers and testers.