Abstract
Measuring Efficiency in neural network system development is an open research problem. This paper presents an experimental framework to measure the training efficiency of a neural architecture. To demonstrate our approach, we analyze the training efficiency of Convolutional Neural Networks and Bayesian equivalents on the MNIST and CIFAR-10 tasks. Our results show that training efficiency decays as training progresses and varies across different stopping criteria for a given neural model and learning task. We also find a non-linear relationship between training stopping criteria, training Efficiency, model size, and training Efficiency. Furthermore, we illustrate the potential confounding effects of overtraining on measuring the training efficiency of a neural architecture. Regarding relative training efficiency across different architectures, our results indicate that CNNs are more efficient than BCNNs on both datasets. More generally, as a learning task becomes more complex, the relative difference in training efficiency between different architectures becomes more pronounced.
| Original language | English |
|---|---|
| Article number | 349 |
| Journal | Artificial Intelligence Review |
| Volume | 57 |
| Issue number | 12 |
| DOIs | |
| Publication status | Published - Dec 2024 |
Keywords
- Deep learning
- Deep neural networks
- Efficiency
- Hyperparameters
Fingerprint
Dive into the research topics of 'A framework for measuring the training efficiency of a neural architecture'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver