Machine Learning-Representation Learning-Deep Learning

1. machine learningmachine learning

Machine learning is to learn certain rules from a limited data set, and then apply the learned rules to some similar sample sets to make predictions. The history of machine learning can be traced back to the artificial neuron network proposed by McCulloch in the 1940s. Currently, the academic community roughly divides machine learning into two categories: traditional machine learning and machine learning. Among them, traditional machine learning focuses more on feature extraction (extracting effective features from the data set) and feature transformation (performing certain processing on the extracted features) problem, that is, more human intervention is required by experts in a certain field. At this time, many traditional machine learning problems becomefeature engineering.

Traditional machine learning process:

1. Data preprocessing: Mainly to remove data set noise
2. Feature extraction: It mainly extracts effective features from the denoised data set. Different measures are taken according to different problems. For example, in image problems, SIFT is generally used. The (scale invariant feature transform) method extracts features with unchanged edges and the same size; in some problems, the variables that have the greatest impact on the research problem may be directly selected as candidate features
3. Feature transform: It mainly involves some form of further integration and screening of the extracted effective features, and finally determines the features that have the greatest impact on the research problem. For example, PCA (principal component analysis) method, RF (random forest) method, etc.
4. Prediction: Use the finalized features to predict the new sample values ​​corresponding to these features to obtain the prediction results of the research problem

But if we don’t have professional knowledge, how to improve the accuracy of prediction results? This leads to the emergence of representation learning (Representation Learning)

2. Representation Learning

To understand representation learning, you must first clarify a concept, Semantic gap: refers to input data and final output Inconsistency in data semantic information.
For example, input many "pleasure" speeches of different people. Because each person's way of speaking is different, each speech expresses "pleasure" in a different way, so If the model is only allowed to learn one or more speech expressions of "pleasure", the model is likely not to know the ways of expressing "pleasure" in speech that has not been learned. This will lead to the problem that if it is given a new speech (new input data), then the model cannot make accurate "pleasure" predictions (output data), that is, the model's generalization ability may not be ideal.

This is the main reason why representation learning and deep learning emerged.

Representation learning: A method that can automatically learn effective features and ultimately improve the performance of machine learning models.
In order to solve the semantic gap, representation learning performs "representation processing" on the input data features so that the model can learn more representative features. There are two ways: < /span>embedding. The process of converting local representation into distributed representation is calledDistributed representation and Local representation (commonly one-hot form)

Both local representation and distributed representation appear in the form of vectors, but distributed representation has better performance, low dimensionality, and efficient data processing
Another key to representation learning is to build a A certain depth ofmulti-level feature representation, which promotes the emergence of deep learning

3. Deep LearningDeep Learning

Deep learning is a sub-problem of machine learning. Its main purpose is to automatically learn effective feature representations from input data. Is it similar to representation learning?

The essence of deep learning is representation learning. At the same time, deep learning further solves the semantic gap problem by adding mid-level features to the input data and input data, which means that the features of the input data undergo many non-linear transformations.

Deep learning mainly adopts the neural network model structure. The parameters in the network model are mainly automatically adjusted through the back propagation algorithm to achieve the effect of automatically learning effective features and improving the prediction ability of the model.

A Machine Learning VS Representation Learning

Reference 1-Representation Learning & Feature Learning
Reference 2-Representation Learning

• Feature engineering: Relying on experts to extract and display features requires a huge amount of work. The quality of feature selection will directly determine the quality of data representation, thus affecting the performance of subsequent tasks.
• Representation learning: The model is used to automatically learn the implicit features of the data. Data representation and subsequent tasks are often jointly trained. It does not rely on expert experience, but requires a larger training data set.
  

B Machine Learning VS Deep Learning

Summarize

The essence of machine learning is to use machines to automatically (with some human participation) learn data features to predict specific tasks for similar inputs (same features). If someone is involved in feature processing, we call it feature engineering;
We can use representation learning and deep learning methods to make the machine learning model achieve the feature rules of the model itself, thereby independently improving the model performance Effect.