版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/chenKFKevin/article/details/79253042
K近邻模型本身非常直观并且容易理解。算法描述起来也很简单，如下图所示。假设我们有一些携带分类标记的训练样本，分布于特征空间中；蓝色、绿色的样本各自代表其类别。对于一个待分类的红色测试样本点，未知其类别，按照成语“近朱者赤，近墨者黑”的说法，我们需要寻找与这个待分类的样本在特征空间中距离最近的K个已标记样本作为参考，来帮助我们做出分类决策。这便是K近邻算法的通俗解释。而在下图中，如果我们根据最近的K=3个带有标记的训练样本做分类决策，那么待测试的样本应该属于绿色级别，因为在3个最近邻的已标记样本中，绿色类别样本的比例最高；如果我们扩大搜索范围，设定K=7，那么分类器则倾向待测样本属于蓝色。因此我们也可以发现，随着K的不同，我们会获得不同效果的分类器。
这里我们将展示如何利用K近邻算法对生物物种进行分类，并且使用最为著名的“鸢尾”Iris数据集。下面的代码是如何读取该数据。
Notes
-----
Data Set Characteristics:
:Number of Instances: 150 (50 in each of three classes)
:Number of Attributes: 4 numeric, predictive attributes and the class
:Attribute Information:
- sepal length in cm
- sepal width in cm
- petal length in cm
- petal width in cm
- class:
- Iris-Setosa
- Iris-Versicolour
- Iris-Virginica
:Summary Statistics:
============== ==== ==== ======= ===== ====================
Min Max Mean SD Class Correlation
============== ==== ==== ======= ===== ====================
sepal length: 4.3 7.9 5.84 0.83 0.7826
sepal width: 2.0 4.4 3.05 0.43 -0.4194
petal length: 1.0 6.9 3.76 1.76 0.9490 (high!)
petal width: 0.1 2.5 1.20 0.76 0.9565 (high!)
============== ==== ==== ======= ===== ====================
:Missing Attribute Values: None
:Class Distribution: 33.3% for each of 3 classes.
:Creator: R.A. Fisher
:Donor: Michael Marshall (MARSHALL%[email protected])
:Date: July, 1988
This is a copy of UCI ML iris datasets.
http://archive.ics.uci.edu/ml/datasets/Iris
The famous Iris database, first used by Sir R.A Fisher
This is perhaps the best known database to be found in the
pattern recognition literature. Fisher's paper is a classic in the field and
is referenced frequently to this day. (See Duda & Hart, for example.) The
data set contains 3 classes of 50 instances each, where each class refers to a
type of iris plant. One class is linearly separable from the other 2; the
latter are NOT linearly separable from each other.
References
----------
- Fisher,R.A. "The use of multiple measurements in taxonomic problems"
Annual Eugenics, 7, Part II, 179-188 (1936); also in "Contributions to
Mathematical Statistics" (John Wiley, NY, 1950).
- Duda,R.O., & Hart,P.E. (1973) Pattern Classification and Scene Analysis.
(Q327.D83) John Wiley & Sons. ISBN 0-471-22361-1. See page 218.
- Dasarathy, B.V. (1980) "Nosing Around the Neighborhood: A New System
Structure and Classification Rule for Recognition in Partially Exposed
Environments". IEEE Transactions on Pattern Analysis and Machine
Intelligence, Vol. PAMI-2, No. 1, 67-71.
- Gates, G.W. (1972) "The Reduced Nearest Neighbor Rule". IEEE Transactions
on Information Theory, May 1972, 431-433.
- See also: 1988 MLC Proceedings, 54-64. Cheeseman et al"s AUTOCLASS II
conceptual clustering system finds 3 classes in the data.
- Many, many more ...
特点分析：K近邻（分类）是非常直观的机器学习模型，因此深受广大初学者的喜爱。许多教科书常常以此模型为例抛砖引玉，便足以看出其不仅特别，而且尚有瑕疵之处。其实跟前篇比起来，你能发现K近邻算法与其他模型最大的不同在于：该模型没有参数训练过程。也就是说我们并没有通过任何学习算法分析训练数据，而只是根据测试样本在训练数据的分布直接作出分类决策。因此，K近邻属于无参数模型中非常简单一种。然而，正是这样的决策算法，导致了非常高的计算复杂度和内存消耗。因为该模型每处理一个测试样本，都需要对所有预先加载在内存的训练样本进行遍历、逐一计算相似度、排序并且选取K个最近邻训练样本的标记，进而做出分类决策。这是平方级别的算法复杂度，一旦数据规模稍大，使用者便需要权衡更多计算时间的代价。
这里我们将展示如何利用K近邻算法对生物物种进行分类，并且使用最为著名的“鸢尾”Iris数据集。下面的代码是如何读取该数据。
# 从sklearn.datasets 导入 iris数据加载器。
from sklearn.datasets import load_iris
# 使用加载器读取数据并且存入变量iris。
iris = load_iris()
# 查验数据规模。
iris.data.shape
(150L, 4L)
# 查看数据说明。对于一名机器学习的实践者来讲，这是一个好习惯。
print iris.DESCRNotes
-----
Data Set Characteristics:
:Number of Instances: 150 (50 in each of three classes)
:Number of Attributes: 4 numeric, predictive attributes and the class
:Attribute Information:
- sepal length in cm
- sepal width in cm
- petal length in cm
- petal width in cm
- class:
- Iris-Setosa
- Iris-Versicolour
- Iris-Virginica
:Summary Statistics:
============== ==== ==== ======= ===== ====================
Min Max Mean SD Class Correlation
============== ==== ==== ======= ===== ====================
sepal length: 4.3 7.9 5.84 0.83 0.7826
sepal width: 2.0 4.4 3.05 0.43 -0.4194
petal length: 1.0 6.9 3.76 1.76 0.9490 (high!)
petal width: 0.1 2.5 1.20 0.76 0.9565 (high!)
============== ==== ==== ======= ===== ====================
:Missing Attribute Values: None
:Class Distribution: 33.3% for each of 3 classes.
:Creator: R.A. Fisher
:Donor: Michael Marshall (MARSHALL%[email protected])
:Date: July, 1988
This is a copy of UCI ML iris datasets.
http://archive.ics.uci.edu/ml/datasets/Iris
The famous Iris database, first used by Sir R.A Fisher
This is perhaps the best known database to be found in the
pattern recognition literature. Fisher's paper is a classic in the field and
is referenced frequently to this day. (See Duda & Hart, for example.) The
data set contains 3 classes of 50 instances each, where each class refers to a
type of iris plant. One class is linearly separable from the other 2; the
latter are NOT linearly separable from each other.
References
----------
- Fisher,R.A. "The use of multiple measurements in taxonomic problems"
Annual Eugenics, 7, Part II, 179-188 (1936); also in "Contributions to
Mathematical Statistics" (John Wiley, NY, 1950).
- Duda,R.O., & Hart,P.E. (1973) Pattern Classification and Scene Analysis.
(Q327.D83) John Wiley & Sons. ISBN 0-471-22361-1. See page 218.
- Dasarathy, B.V. (1980) "Nosing Around the Neighborhood: A New System
Structure and Classification Rule for Recognition in Partially Exposed
Environments". IEEE Transactions on Pattern Analysis and Machine
Intelligence, Vol. PAMI-2, No. 1, 67-71.
- Gates, G.W. (1972) "The Reduced Nearest Neighbor Rule". IEEE Transactions
on Information Theory, May 1972, 431-433.
- See also: 1988 MLC Proceedings, 54-64. Cheeseman et al"s AUTOCLASS II
conceptual clustering system finds 3 classes in the data.
- Many, many more ...
通过上述代码对数据的查验以及数据本身的描述，我们了解到Iris数据集共有150朵鸢尾数据样本，并且均匀分布在3个不同的亚种；每个数据样本被4个不同的花瓣、花萼的形状特征所描述。由于没有指定的测试集，因此按照惯例，我们需要对数据进行随机分割，25%的样本用于测试，其余75%的样本用于模型的训练。
# 从sklearn.cross_validation里选择导入train_test_split用于数据分割。
from sklearn.cross_validation import train_test_split
# 从使用train_test_split，利用随机种子random_state采样25%的数据作为测试集。
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.25, random_state=33)# 从sklearn.preprocessing里选择导入数据标准化模块。
from sklearn.preprocessing import StandardScaler
# 从sklearn.neighbors里选择导入KNeighborsClassifier，即K近邻分类器。
from sklearn.neighbors import KNeighborsClassifier
# 对训练和测试的特征数据进行标准化。
ss = StandardScaler()
X_train = ss.fit_transform(X_train)
X_test = ss.transform(X_test)
# 使用K近邻分类器对测试数据进行类别预测，预测结果储存在变量y_predict中。
knc = KNeighborsClassifier()
knc.fit(X_train, y_train)
y_predict = knc.predict(X_test)
与之前的评价指标一样，我们使用准确性、召回率、精确率和F1指标，这4个测度对K近邻分类模型在经典鸢尾花品种预测任务上进行性能评估，详细代码如下：
# 使用模型自带的评估函数进行准确性测评。
print 'The accuracy of K-Nearest Neighbor Classifier is', knc.score(X_test, y_test)
The accuracy of K-Nearest Neighbor Classifier is 0.894736842105
# 依然使用sklearn.metrics里面的classification_report模块对预测结果做更加详细的分析。
from sklearn.metrics import classification_report
print classification_report(y_test, y_predict, target_names=iris.target_names)
precision recall f1-score support
setosa 1.00 1.00 1.00 8
versicolor 0.73 1.00 0.85 11
virginica 1.00 0.79 0.88 19
avg / total 0.92 0.89 0.90 38特点分析：K近邻（分类）是非常直观的机器学习模型，因此深受广大初学者的喜爱。许多教科书常常以此模型为例抛砖引玉，便足以看出其不仅特别，而且尚有瑕疵之处。其实跟前篇比起来，你能发现K近邻算法与其他模型最大的不同在于：该模型没有参数训练过程。也就是说我们并没有通过任何学习算法分析训练数据，而只是根据测试样本在训练数据的分布直接作出分类决策。因此，K近邻属于无参数模型中非常简单一种。然而，正是这样的决策算法，导致了非常高的计算复杂度和内存消耗。因为该模型每处理一个测试样本，都需要对所有预先加载在内存的训练样本进行遍历、逐一计算相似度、排序并且选取K个最近邻训练样本的标记，进而做出分类决策。这是平方级别的算法复杂度，一旦数据规模稍大，使用者便需要权衡更多计算时间的代价。