Online Survey Fraud: Behavioural Traces of Bots

Abstract

As bots become more sophisticated, they pose growing challenges and undermine research validity. Current automated detection tools, such as CAPTCHA and QualtricsRelevantID functions, are becoming increasingly unreliable, leaving a methodological gap in understanding and addressing survey fraud. To address this gap, this study investigates fraudulent responses in online surveys by combining Digital Behavioural Evidence (DBE) with criminology theories to distinguish between genuine and fraudulent responses. It applies a case- control exploratory observational design on secondary data provided by the Queensland Drug Checking Services Project, which used two recruitment strategies: an on-site QR code poster and a publicly shared social media URL. Paradata, such as IP addresses, timestamps, and email structures, are investigated using manual review, which includes point-based scoring, temporal and pattern analysis, and qualitative review. The results are then interpreted through relevant criminological theories to help make sense of the behaviours and motivations linked to survey fraud. Results show a strong contrast between the control QR responses, which reflected legitimate human variation, and the URL responses, which showed systematic anomalies. Automated systems flagged 25% (n = 99) as fraud once activated, while manual DBE review identified 98% of the entire URL dataset (n = 314) as fraud. The study demonstrates that behaviourally grounded, theoretically informed fraud detection methods can enhance detection and offer recommendations to improve recruitment strategies, preventive measures, and automated systems.

Keywords: Digital Behavioural Evidence (DBE); Online Survey Fraud; Criminology Theory; Automated Bot Detection; Qualtrics; CAPTCHA