Deep Learning and Transfer Learning for Automated Image-Based Classification of Oral Epithelial Dysplasia: A Comparative Study of Convolutional Neural Network Architectures

Background: Oral epithelial dysplasia (OED) is a spectrum of any architectural and cytological abnormality with an increased potential for malignant transformation. While the histopathological assessment continues to be a key element in the diagnosis and grading process, there is a risk of some subjectivity due to the complexity of morphological features and interobserver variability. With the rise of the digital pathology, incorporation of deep-learning algorithms for automated analysis of histopathological images has become possible. Convolutional neural networks (CNNs) are able to learn hierarchical morphological representations directly from image data in contrast to conventional machine-learning approaches that rely on manually selected morphometric variables. Material and Methods: A total of 81 histopathologically evaluated specimens were included, comprising 27 NBM, 27 LR, and 27 HR cases. Representative histopathological images were digitally acquired under standardized microscopic conditions and pre-processed for computational analysis. A 12-layer CNN was developed for binary classification, while a separate 13-layer CNN was developed for multiclass classification of NBM, LR, and HR. Transfer-learning experiments were subsequently performed using pretrained ResNet-50, VGG16, VGG19, MobileNet-V2, and Inception-V3 architectures. Model performance was assessed using accuracy, precision, recall, and F1-score. Results: The proposed multiclass CNN achieved an overall classification accuracy of 89.74%. The HR group demonstrated the strongest class-specific performance, with a precision of 0.92, recall of 0.94, and F1-score of 0.93. The LR group demonstrated comparatively lower performance, with precision, recall, and F1-score of 0.76, 0.71, and 0.73, respectively. Among the evaluated transfer-learning architectures, ResNet-50 demonstrated the highest multiclass accuracy (91.86%), followed by VGG19 (88.53%), VGG16 (86.92%), Inception-V3 (84.75%), and MobileNet-V2 (83.68%). Conclusion: The results showed that the deep-learning image analysis algorithm could be used to automatically classify OED and was feasible. The transfer learning, especially ResNet-50, showed good classification results in the training set. While deep-learning methods could be used as computational aids for objective evaluation of dysplastic lesions, further larger multicentric data sets, case-level validation, independent external testing and prospective clinical evaluation are needed to implement in clinical practice.

Keywords: Deep Learning, convolutional neural network, transfer learning, ResNet-50, oral epithelial dysplasia, oral leukoplakia, digital pathology, AI, histopathology, oral potentially malignant disorders.