Malignant Transformation of Oral Epithelial Dysplasia: Precision Diagnostics Utilizing a Deep Learning and Spatial Transcriptomics Predictive Modeling Approach.

Abstract

Oral squamous cell carcinoma (OSCC) remains a major global health burden with limited improvements in overall survival over recent decades. Most OSCCs arise from oral potentially malignant disorders (OPMDs), including oral epithelial dysplasia (OED), which is currently graded subjectively by histopathological examination. The urgent need for objective, biologically informed risk stratification tools has driven the integration of artificial intelligence (AI), spatial transcriptomics, and functional genomics in oral cancer research. This thesis tests the central hypothesis that deep learning and spatial transcriptomic approaches can objectively predict the malignant transformation of OED by identifying histomorphological patterns and immune-epithelial gene signatures associated with malignant transformation zones and cancer progression. To evaluate this, three specific aims were pursued: 1. Develop and compare AI models for predicting malignant transformation of OED based on lymphocyte distribution and tissue morphology. 2. Identify spatially informed predictive biomarkers in proliferative leukoplakia (PL) using spatial transcriptomic profiling. 3. Functionally assess the role of mEAK-7, a novel regulator of non-canonical mTOR signaling, in OSCC initiation using a gene knockout mouse model. In Aim 1, we trained and evaluated multiple machine learning and deep learning models, including classical regressors, state-of-the-art neural networks, and weakly supervised pattern-recognition networks using a multi-institutional dataset of annotated whole slide images (WSIs) of OPMD cases with known transformation status. In Aim 2, spatial transcriptomics (10x Genomics Visium HD) was performed on PL samples to identify gene signatures predictive of transformation, with a focus on immune–epithelial interactions. In Aim 3, a 4NQO-induced oral carcinogenesis model was applied to mEAK- 7 knockout mice to assess its functional role in OSCC development. AI models demonstrated that lymphocyte infiltration patterns can predict malignant transformation, with deep learning models achieving accuracies up to 83.4% in distinguishing transformed from non-transformed cases. Spatial transcriptomics revealed downregulation of epithelial barrier genes (FLG, CASP14) and immune activation signatures (S100A8, S100A9, CD74) in transformation zones, supporting a model of barrier disruption and neoantigen-driven immune remodeling. The mEAK-7 knockout study showed significantly reduced OSCC incidence, implicating alternative mTOR signaling in OSCC initiation and validating spatial findings through in vivo functional evidence. In conclusion, this thesis presents an integrated, multi-modal investigation into the malignant transformation of OED, providing evidence that AI and spatial biology can complement conventional pathology in predicting cancer risk. The combined findings offer a foundation for future precision diagnostics in oral cancer prevention and identify novel molecular targets for early intervention.