Neural Networks based on Feed Forward architectures (FFN) have found a wide range of real world applications, with unprecedented capabilities for image and speech recognition tasks. Despite these empirical successes, a theoretical understanding of the underlying design principles is still limited. Finding the correct number of layers and units in a FFN seems to be a matter of trial and error rather than a well defined scientific task. Same is true for devising novel architectures or to optmize existing ones. …

In this post I will consider a problem from combinatorial optimization, that can be understood as constrained optimization on a general graph. This gives me the opportunity to introduce the concepts and language of complex networks in a more general way than those usually involved in Neural Networks alone. The concepts developed here will be used in a subsequent series on the physics of Neural Networks models.

Rather than discussing network theory from the most general point of view, I will focus here on solving a particular problem, from setting up the mathematical model to its numerical implementation using Google’s…