The subject of Neural Networks is being seen to be coming of age, after its initial inception 50 years ago in the seminal work of McCulloch and Pitts. It is proving to be valuable in a wide range of academic disciplines and in important applications in industrial and business tasks. The progress being made in each approach is considerable. Nevertheless, both stand in need of a theoretical framework of explanation to underpin their usage and to allow the progress being made to be put on a firmer footing.This book aims to strengthen the foundations in its presentation of mathematical approaches to neural networks. It is through these that a suitable explanatory framework is expected to be found. The approaches span a broad range, from single neuron details to numerical analysis, functional analysis and dynamical systems theory. Each of these avenues provides its own insights into the way neural networks can be understood, both for artificial ones and simplified simulations. As a whole, the publication underlines the importance of the ever-deepening mathematical understanding of neural networks.