Foreword
It can not be called more than familiar with this field, after all, no systematic research. Recently I read two papers, right when the summary.
Learning Optimizer
Summary: In this method, a network (meta learner meta-learner) Learning updates another network (Learner learner), in order to efficiently learner learning task. In order to better optimize the neural network, the people of this method has been extensively studied. Meta-learning is usually a circulation network, in order to keep in mind is how to correct before the learner model.
Thesis: OPTIMIZATION AS A MODEL FOR FEW- SHOT LEARNING, ICLR 2017
innovations: propose a LSTM-based model of meta-learner, learner learning optimization to improve the classification performance in the few shot field.
- What is a few shot learning? Literally see, the lens is less task to learn. What does it mean? Before we always have a lot of classification task of data collection, in fact, we humans do not need to learn vast amounts of data, such as humans may see dogs and cats to a few photos, you can judge a dog or cat belongs to a new photo. this human behavior can truly be called intelligent. few shot is learning to solve a small data set. in this paper, we trying to learn how to classify data from very few marks in. we have a collection of data (darasets), but each type of data which are only a few samples (example)
- What is a meta-learning? The above problems are divided into two learning tasks, the first task is to focus on individual data fast access to knowledge, the second task is to guide the first task, that is, slowly extract all the learning tasks to the essence. metamodel to better generalization and adapt to new problems.
- Gradient descent small sample of the problem. A small sample means that a limited number of gradient descent, for the case of non-convex, the resulting model necessarily poor performance. Furthermore, for each individual data sets, each neural network are is the random initialization, after several iterations converge to optimal performance is difficult.
The figure represents the meta-learning training constitutes Here is a 1 shot 5way way. There are five categories for each training set images, only one sample of each type of picture.
The key of this paper is how to update the gradient descent and LSTM link
Training samples are left broken line, the right side is the test sample. LSTM gradient update occurs after the test sample is generated loss.
