Designing Compact Neural Architectures via Neuron Gating and Mixed Activation Titelbild

Designing Compact Neural Architectures via Neuron Gating and Mixed Activation

Designing Compact Neural Architectures via Neuron Gating and Mixed Activation

Jetzt kostenlos hören, ohne Abo

Details anzeigen
Neural Architecture Search is powerful but expensive due to discrete, combinatorial design choices. This paper proposes continuous relaxations of neuron-level and activation-level decisions, enabling fully differentiable optimization across MLPs, CNNs, RNNs, and Transformers. Three resulting methods (NAS-NG, NAS-MA, NAS-NGMA) find highly compact architectures --- including a CNN with just 0.26M parameters hitting 99.63% MNIST accuracy --- while outperforming standard DARTS on CIFAR-10. This offers a scalable, general-purpose toolkit for automatically designing efficient models, valuable for deploying AI on resource-constrained devices like mobile phones or edge hardware. Authors: Abhishek Shukla, Ankur Sinha, Faiz Hamid Paper: https://arxiv.org/abs/2608.14443v1
adbl_web_anon_alc_button_suppression_t1
Noch keine Rezensionen vorhanden