At International Conference on Artificial Neural Networks (ICANN 2018) held in Rhodes, Greece, I presented following two topics.
“Check Regularization: Combining Modularity and Elasticity for Memory Consolidation”
“Practical Fractional-Order Neuron Dynamics for Reservoir Computing”
The first topic proposed a new regularization method to mitigate the catastrophic forgetting of neural network. Specifically, I combined the elasticity to the optimal parameters for old tasks and the sparsity to make a room for new tasks. To combine these two abilities, I designed the boundary between them depending on the importance of the parameters for old tasks.
The second topic proposed a new neuron dynamics in reservoir computing to obtain long memory capacity. Specifically, I focused on the fractional-order leaky integrator, which needs all the past states. To exploit this dynamics in practice, I approximated it reasonably to get the constant calculation cost.