On October 4th, Kaitlin Maile successfully defended her PhD, entitled Dynamic architectural optimization of artificial neural networks (abstract below). Kaitlin was my first PhD student to defend, and I was very lucky to have been able to advise her. Kaitlin and I started collaborating during her Master’s, when she decided to come to Toulouse to work with me on bio-inspired methods for learning. She has an intense drive to push neural networks to perform better and more like the human brain, which they’re still behind in many ways. She applied this idea to networks of all scales, from toy MLPs to giant transformer models, showing how ANNs can be more plastic while learning, as they are in the human brain. In a time when companies are spending so much money and energy on training new models from scratch, her work on growing models couldn’t be more relevant; I’m sure she’ll go on to do great things!
Dynamic architectural optimization of artificial neural networks
Artificial neural networks have fundamentally redefined the way data are analyzed and opened new artificial intelligence possibilities through the field of deep learning. Despite this progress, artificial neural networks still have many limitations that are not seen in biological neural networks such as catastrophic forgetting and extreme specialization. The idea of this project is to apply the learning principles observed in biological neurons to artificial neural networks, including synaptic plasticity and structural learning. The aim is that this will allow neural networks to adapt their structure and learning to specific tasks and to learn continuously without forgetting prior knowledge. The specific objective of this project is to improve reinforcement learning methods, but this fundamental work on artificial neural network learning has potential in all application areas where artificial neural networks are currently applied and could lead to further applications.