Multi-Dataset Dermatological Lesion Classification via Self-Supervised Feature Extraction and Attention-Driven Hybrid Resampling Networks

Automated classification of multi-dataset skin lesions
is constrained by severe class imbalances and poor cross-dataset
generalization across clinical and dermoscopic modalities. To
resolve these gaps, this paper presents a unified framework com-
bining deep self-supervised feature extraction with an attention-
enhanced hybrid resampling network. We utilize a frozen DI-
NOv2 Vision Transformer backbone to extract 384-dimensional
continuous image embeddings without specialized fine-tuning.
To eliminate dataset skewness, the continuous vector space
is balanced via a hybrid SMOTETomek resampling protocol,
followed by a downstream Multi-Head Attention neural network
that dynamically weights fine-grained pathological variations.
Rigorous stratified 5-fold cross-validation across four landmark
datasets (HAM10000, Skin Disease, ISIC 2019, and DermNet), en-
compassing over 70,000 images spanning 7 to 23 distinct classes,
validates our architecture against traditional machine learning
baselines. The proposed model demonstrates high computational
stability, yielding hold-out test accuracies ranging from 72.41% to
98.00% and macro-averaged AUC–ROC scores between 96.32%
and 99.88%. Notably, on the highly complex 22-class Skin Disease
hold-out pool of 12,108 images, the framework records a robust
accuracy of 84.07%, with a Cohen’s Kappa of 0.8332 and a
Matthews Correlation Coefficient (MCC) of 0.8332. This study
highlights the potential of foundation models with structural
attention to support clinical decision-making, reduce diagnostic
delays, and improve equitable access to advanced dermatological
identification in resource-constrained environments.