A Review of Parallel Mappings for Feed Forward Neural Networks using the Backpropagation Learning Algorithm

The Back-Propagation (BP) Neural Network (NN) is probably the most well known of all neural networks. Many mappings of the BP NN have been implemented for both special purpose and general-purpose computers. This mapping process can fall into one of two categories: heuristic mapping and algorithmic mapping. Heuristic mapping concentrates on the architecture and behaviour of the network as it trains whereas algorithmic mapping concentrates on the parallelization of the learning algorithm. Mappings in the heuristic category tend to take a trial and error approach based on the understanding of the network and of the target machine. In comparison mappings in the algorithmic category tend to take a more theoretical approach to the parallelization process. A number of heuristic mapping schemes exist for BP networks so it is worthwhile to investigate their strengths and weaknesses.