Review of the Statistical Errors in Saudi Medical Journal

Abstract

Introduction Statistical analysis is a vital part of most biomedical research articles as it aids in understanding and interpreting the findings of the research. However, several studies have found that it is often misused in the biomedical literature. In this study, we reviewed the statistical analysis of a national journal to shed the light on low impact factor journals. As a secondary objective, we reviewed the statistical analysis of a recent article from a journal with high impact factor. Method A reporting checklist was developed to investigate the effectiveness of the statistical reporting. Comprehensive checklists for t-test and ANOVA statistical review were developed based on previously published checklists. The high impact factor journal was chosen to be Immunity and the Saudi Medical Journal (SMJ) was chosen as the lower impact factor journal. Results & Discussion In the chosen article from Immunity journal, the reporting of statistics was inadequate as it involved 8 statistical errors. In addition, the t-test was used for more than 2 groups without adjusting for multiple comparisons. The review of SMJ included 19 original articles published in August and September 2020. The reporting of statistics was generally inadequate as all studies had at least three reporting errors. Nonetheless, some investigated errors were minor and were unlikely to affect the conclusion of the studies. Only two studies had major issues where their conclusion was deemed to be inappropriate. The performance of t-test and ANOVA was found to be appropriate in all relevant SMJ studies which is much better than previously reported. Conclusion The statistical reporting and errors rate in SMJ are not satisfactory. However, the use of t-test and ANOVA in the SMJ seem to be better than previously investigated journals. Despite all the efforts to improve statistics and the many articles discussing the problem of using t-test for more than 2 groups, it is still being used in prestigious journals such as Immunity. Further investigation is needed, involving a larger number of studies and journals, to truly assess the nature and extent of these issues.