1. Feature selection
1. For a learning task, given a set of attributes, some of the attributes are critical and useful. Some properties are useless. Attributes become "features", those that are useful for learning tasks become "relevant features", and those that are not useful become "irrelevant features".
"Feature selection": The process of selecting a subset of relevant features from a given set of features is called "feature selection". - is a very important " data preprocessing ".
2. Why feature selection: (1) There are too many attributes, and important features need to be selected from them.
(2) Remove irrelevant features and reduce the difficulty of learning tasks.
3. Redundant features: This type of feature means that the information it contains can be inferred from other features, which is often useless. For example: for a cube, the feature of "base area" can be deduced from "base length" and "base width", and "base area" is a redundant feature. But redundant features are useful if they happen to be "intermediate concepts" needed to accomplish the task.
4. Feature selection includes: selecting a feature subset that contains all important information from an initial feature set is divided into two steps:
(1) " Subset search": Select candidate sets from the given feature set {d1, d2, d3...dn}, which are divided into two:
"Forward search": first regard each feature {d1},{d2}...{dn} as a candidate subset, evaluate , select the best one, and then add a feature in the next round to form Contains a subset of two features, select the best among these n-1, ... If in the K+1 round, the best candidate set steps into the previous round, stop generating the candidate set, and in the K+1 round The set of K features selected in the previous round as a consequence of feature selection.
"Backward search": first treat the entire feature {d1, d2...dn} as a candidate subset, evaluate it, and try to remove an irrelevant feature each time.
(2) "Subset evaluation ": the information gain of attribute subsets .
The larger the information gain, the more useful information the feature subset contains.
5. Common feature selection methods can be divided into three categories: filtering, wrapping, and embedded.
(1) Filtered selection: directly perform feature selection without considering subsequent learners
Relief is a well-known filtered feature selection method. It is necessary to formulate a relevant statistic to measure, that is, specify a threshold T, and select the feature corresponding to the relevant statistical component larger than T.
(2) Wrapped selection: The performance of the final learner to be used is directly used as the evaluation criterion for feature subsets, which is the biggest difference from the filtering feature selection method .