In the Deep Learning (DL) age, more and more people have encountered and used (knowingly or not) random matrices. Most of the time this use is limited to the initialization of the networks weights, that can be accomplished with a single line of code in your favorite DL framework. However, Random Matrices have a rich mathematical theory with far reaching applications in physics, network theory, machine learning, finance, etc. This fascinating range of applications also means that each field often developed its dedicated terminology to describe the same mathematical concept, often with confusing consequences. The aim of this article(s) is…

Neural Networks based on Feed Forward architectures (FFN) have found a wide range of real world applications, with unprecedented capabilities for image and speech recognition tasks. Despite these empirical successes, a theoretical understanding of the underlying design principles is still limited. Finding the correct number of layers and units in a FFN seems to be a matter of trial and error rather than a well defined scientific task. Same is true for devising novel architectures or to optmize existing ones. …

In this post I will consider a problem from combinatorial optimization, that can be understood as constrained optimization on a general graph. This gives me the opportunity to introduce the concepts and language of complex networks in a more general way than those usually involved in Neural Networks alone. The concepts developed here will be used in a subsequent series on the physics of Neural Networks models.

Rather than discussing network theory from the most general point of view, I will focus here on solving a particular problem, from setting up the mathematical model to its numerical implementation using Google’s…

Theoretical Physicist, Machine learning scientist@Microsoft, everything Geek.