Synaptic metaplasticity for image processing enhancement in convolutional neural networks

Abstract

Synaptic metaplasticity is a biological phenomenon shortly defined as the plasticity of synaptic plasticity, meaning that the previous history of the synaptic activity determines its current plasticity. This phenomenon interferes with some of the underlying mechanisms that are considered important in memory and learning processes, such as long-term potentiation and long-term depression. In this work, we provide an approach to include metaplasticity in convolutional neural networks to enhance learning in image classification problems. This approach consists of including metaplasticity as a weight update function in the backpropagation stage of convolutional layers. To validate this proposal, we have been used eight different award-winning convolutional neural networks architectures: LeNet-5, AlexNet, GoogLeNet, VGG16, VGG32, ResNet50, DenseNet121 and DenseNet169; trained with four different popular datasets for benchmarking: MNIST, Fashion MNIST, CIFAR-10 and CIFAR-100. Experimental results show that there is a performance enhancement for each of the convolution neural network architectures in all the datasets used.