Applying Systems Analysis and Decision Support for Improving Feedback Processes in Computing Disciplines

Abstract

Systems are interconnected processes that facilitate and support any work environment, in addition to serving stakeholder and helping them make appropriate decisions. Developing such systems requires improving the processes by analysing their components, identifying strengths and weaknesses with the involvement of stakeholders, and then regulating them. This approach is known in information systems research as systems analysis, development, and design. This study applied this aspect in higher education system, specifically within computer science specialisations. We focused on one of the most important processes in the educational system: the feedback process.

Systems in higher education consist of interconnected processes to support students' academic and professional improvement. It also helps and supports making appropriate decisions that serve the educational process's interest. Feedback is one of the essential processes in that system because it plays a crucial role in student learning, fostering more profound understanding and knowledge and promoting meta-cognition. It is vital to ensure that feedback processes adapt to the continuously changing teaching environment to benefit students.

Due to the dual nature of computing education, combining theoretical knowledge and practical skill development, feedback in this discipline is especially important. In addition, computing education requires students to engage in open-ended problem-solving, debugging, and iterative development, demanding feedback beyond simple error correction.

Previous studies have investigated feedback processes in different disciplines and focused on how educators provide feedback, format it, and how students use it. Moreover, these studies focus on improving engagement and effectiveness, but research on feedback in computing education remains limited, particularly from the perspective of computing educators. Furthermore, these studies usually focus on a limited part of the feedback process; they do not combine educators' and students' perspectives and often do not take into account the whole learning environment.

In particular, the feedback process does not consider the computing students' backgrounds, such as students' backgrounds (level of education, industry experience, etc.), students who come from different cultures, etc. Addressing these gaps is essential for developing more effective feedback processes tailored to computing education.

This thesis aims to enhance feedback processes in computing education by analyzing and improving the feedback system. Strengthening these processes will support stakeholders in making appropriate and informed decisions. The rapid evolution of the computing field requires feedback approaches that adapt to new technologies and industry trends. One important factor we investigate in the feedback process is the differences in learning environments, identifying how they influence their effectiveness, particularly from the perspectives of educators and students regarding feedback methods. This study examines how different teaching and learning strategies impact feedback methods and identifies key factors contributing to high-quality feedback. Furthermore, this study aims to propose guidelines for novice computing educators, especially those without prior experience, to help them provide quality feedback while considering different situations. These guidelines are developed based on insights from experienced computing educators and computing students to improve the feedback process and support student learning after understanding all possible influencing factors.

This study's comparative approach is a significant contribution, as it examines feedback processes at two universities from different countries and cultures. Institutional policies, learning environments, and cultural factors shape the effectiveness of feedback. For instance, at Jeddah University, computer science courses are taught in English to Arabic-speaking students, potentially affecting their engagement and comprehension of feedback, whereas RMIT students are either native speakers or have a certified command of English. Considering such linguistic and contextual factors, this research provides a culturally independent perspective on feedback in computing education. The findings will contribute to developing practical recommendations to improve feedback strategies in diverse educational environments.

To achieve these aims, we selected two distinct universities, Jeddah University in Saudi Arabia and RMIT University in Australia. Beyond the cultural differences, these universities also represent different learning environments, including variations in policies and structures. We conducted semi-structured interviews with computing educators from both institutions to explore their perspectives and preferences regarding five main feedback methods: adding feedback directly in the submission file, adding feedback within the rubric, recording videos for feedback, recording audio for feedback and providing feedback through scheduled feedback sessions of various types (group/individual, online/face-to-face).

Moreover, we surveyed computing students from the both universities, asking them the same questions posed to educators. This comparison helped us identify similarities and differences between students' and educators' perspectives, ultimately aiding in developing improved feedback process guidelines that account for both groups' needs, preferences, and perspectives.

After our analysis, we identified educators' perspectives and factors influencing them. We examined computing educators' estimates of the time required to provide effective feedback to their students. Regarding computing students, we investigated the factors affecting their feedback preferences. Based on previous literature, we developed 16 hypotheses: eight related to feedback methods in general and eight specific to feedback sessions. Jeddah University accepted five general feedback method hypotheses, all eight related to feedback sessions. RMIT University accepted six hypotheses concerning general feedback methods, and two hypotheses were related to feedback sessions.

Then, we compared and analysed both cohorts of computing students and computing educators. First, we examined Jeddah University's cohorts, and then we compared RMIT University's cohorts. Lastly, we conducted a general comparison between the computing students and educators from both universities to develop culturally independent guidelines. We formalise three sets of guidelines to help and support computing educators, especially novice ones, to deiced the better and more efficient feedback to their students. Then we provided recommendations for applying these guidelines by integrating and adopting tools within learning management systems (LMS). This approach aims to enhance feedback processes and assist computing educators in making more effective decisions.