NeuroPlast : a learnable activation function evaluated under knowledge distillation for medical image classification

Article Article scientifique Article NeuroPlast : a learnable activation function evaluated under knowledge distillation for medical image classification eng 2026-08-31 2026 author Vaussenat Fabrice Department of Electrical Engineering, École de technologie supérieure, Montréal, QC, Canada. 460362 author Bhattacharya Abhiroop Department of Electrical Engineering, École de technologie supérieure, Montréal, QC, Canada. author Payette Julie Department of Electrical Engineering, École de technologie supérieure, Montréal, QC, Canada. author Desmettre Thibaut Department of Acute Medicine, Hôpital Universitaire de Genève, Genève, Switzerland. 412936 Département d'anesthésiologie, pharmacologie et soins intensifs (APSIC) author Saidi Alireza Institut de recherche Robert-Sauvé en santé et en sécurité du travail (IRSST), Montréal, QC, Canada. author Gagnon Ghyslain Department of Electrical Engineering, École de technologie supérieure, Montréal, QC, Canada. author Cloutier Sylvain G Department of Electrical Engineering, École de technologie supérieure, Montréal, QC, Canada. 1849571 10.3389/frai.2026.1849571 42741410 2624-8212 PMC13572696 https://www.frontiersin.org/articles/10.3389/frai.2026.1849571/full Deep neural networks for medical image classification rely almost exclusively on fixed activation functions such as ReLU. We introduce NeuroPlast, a parametric activation function whose four differentiable components (a shifted sigmoid modeling NMDA-type voltage gating, a Gaussian plateau inspired by AMPA receptor scaling, an excitatory rectifier, and an inhibitory leak) are combined through six learnable parameters. Two mixing strategies are evaluated: a <i>static</i> variant with fixed learned weights, and a <i>metaplastic</i> variant whose mixing coefficients adapt per sample via a lightweight squeeze-excite gate conditioned on channel statistics. NeuroPlast is embedded within NADN Ultra, a 13.5 M-parameter residual convolutional neural network (CNN) with convolutional block attention modules (CBAM), trained entirely from scratch through a two-phase knowledge distillation (KD) pipeline. The teacher is a fine-tuned EfficientNet-B0; the student combines logit-level KD with optional feature-level alignment losses. Across four medical imaging benchmarks (Brain Tumor MRI, 7,200 images, 4 classes; Chest X-ray Pneumonia, 5,856 images, 2 classes; Skin Cancer HAM10000, 10,015 images, 7 classes; COVID-19 Radiography, 10,848 images, 2 classes), evaluated under five-fold stratified cross-validation with 95% confidence intervals, the static-KD variant reaches 99.4-99.5% accuracy on COVID-19 X-ray across three seeds, on par with pretrained EfficientNet-B0 (99.18%, within replication noise) and above ResNet-18 (98.69%). It closes 65% of the accuracy gap on Brain Tumor MRI (98.29% vs. 99.13%), reaches 92.5% on Chest X-ray under a uniform class-balancing rule, and 79.5% on Skin Cancer under lesion-grouped cross-validation. The metaplastic variant achieves 99.28% on COVID-19 X-ray, still above pretrained baselines, but does not consistently outperform the static version, a negative result analyzed through ablation experiments. On three tabular medical datasets spanning three orders of magnitude in sample size (569 to 253,680), NeuroPlast matches five established activations within ±1.7 percentage points; at 253 K samples all activations converge within 0.09 pp. Our findings indicate that knowledge distillation is the primary enabler for from-scratch architectures to approach pretrained-level performance on medical images; under an identical distillation pipeline, NeuroPlast adds a small but consistent gain over ReLU, GELU, Swish, Mish, and PReLU, leading on all four imaging benchmarks by margins below one point. Bio-inspired activation function Convolutional neural network Deep learning From-scratch training Knowledge distillation Learnable activation Medical image classification Metaplasticity Dewey 617 63 Département d'anesthésiologie, pharmacologie, soins intensifs et urgences 229 false Frontiers in artificial intelligence 9 50e7e223-67ae-4724-b6fd-2ce406e2e4c0 Article (Published version) Public Article_scientifique_from_unpaywall.pdf 2536866 application/pdf CC BY-4.0 fa5177c50bc86026f70a6a83a29d94d14457db43b82d399aca070c599a12be6c openAccess Faculté de médecine / Section de médecine clinique / Département d'anesthésiologie, pharmacologie, soins intensifs et urgences

Title	Published in	Access level	OA Policy	Year	Views	Downloads

NeuroPlast : a learnable activation function evaluated under knowledge distillation for medical image classification	Frontiers in artificial intelligence			2026	4	3

Archive ouverte UNIGE

Vaussenat, Fabrice