之前为了熟悉机器学习的东西去搞kaggle的东西，然后就从Titanic入门咯，结果发现并没有对机器学习的东西有深入的理解，做数据挖掘的时候直接调用sklearn里面的框架，根本不用去想机器学习的公式的运用和基础的实现，想用SVM就直接from sklearn import svm，然后clf = svm.SVC(kernel='linear', C=1)往这里面填几个参数，知道基本原理和参数的含义就行。直到我看到这段话：
很多人说深度学习就是个黑箱子，把图像预处理之后丢进 tensorflow 就能出来预测结果，简单有效又省时省力。但正如我在上一篇推送中所说，如果你已是一名功力纯厚的深度学习工程师，这么做当然没问题。但我想大多数人也和我一样，都是走在学习深度学习的路上，一上来就上框架并没有什么特别不妥之处，但总归是对你理解深度学习的黑箱机制是了无裨益的。所以，我建议在学习深度学习的路上，从最简单的感知机开始写起，一步一步捋清神经网络的结构，以至于激活函数怎么写、采用何种损失函数、前向传播怎么写、后向传播又怎么写，权值如何迭代更新，都需要你自己去实现。若在一开始就直接调用框架，小的 demo 可以跑起来，糊弄一时，看起来就像是鸠摩智在内力未到的情形下强行练习少林寺的 72 绝技，最后走火入魔。
我发现我好像走火入魔了，这时我想着我又该把基础打打牢了，之前看过吴老师的机器学习，只能算是理论上有了一定认识，没做过多实践，代码和理论还没好好联系起来，然后就找到了李飞飞深度学习与计算机视觉，下面开始我的第一课——KNN。
先贴点网址
https://zhuanlan.zhihu.com/p/21930884 大牛搞的CS231n官方笔记授权翻译总集
http://cs231n.stanford.edu/2016/syllabus 英文的CS231n官方笔记
http://www.cs.toronto.edu/~kriz/cifar.html 做学习的数据集
特别感谢翻译笔记的哥们。
第一部分 KNN算法实现
先贴原理，具体原理直接看笔记，我自己做个总结
一、概念&意义
二、计算步骤：
1、算距离：给定测试对象，计算它与训练对象距离
2、找邻居：圈定距离最近的 k个对象，作为测试对象的近邻
3、做分类：根据这k个近邻归属的主要类别，来对测试对象分类
三、三个基本要素
I、距离度量
a)当p=2时，为欧式距离：
b)当p=1时，为曼哈顿距离：
II、K值的选择
K值越小，整体模型越复杂，容易发生过拟合
K值越大，整体模型越简单，近似误差会增大（误分类）
III、测试对象类别的判定
a) 多数表决：
其中v是类标号，yi是一个最近邻的类标号，I(.)是指示函数，如果其参数为真，则返回1，否则，返回0
b) 距离加权表决：
# 损失函数的公式L = -(1/N)∑i∑j1(k=yi)log(exp(fk)/∑j exp(fj)) +λR(W)
再上代码
import numpy as np
class KNearestNeighbor(object):#首先是定义一个处理KNN的类
""" a kNN classifier with L2 distance """
def __init__(self):
pass
def train(self, X, y):
self.X_train = X
self.y_train = y
def predict(self, X, k=1, num_loops=0):
if num_loops == 0:
dists = self.compute_distances_no_loops(X)
elif num_loops == 1:
dists = self.compute_distances_one_loop(X)
elif num_loops == 2:
dists = self.compute_distances_two_loops(X)
else:
raise ValueError('Invalid value %d for num_loops' % num_loops)
return self.predict_labels(dists, k=k)
def compute_distances_two_loops(self, X):
num_test = X.shape[0]
num_train = self.X_train.shape[0]
dists = np.zeros((num_test, num_train))
for i in xrange(num_test):
for j in xrange(num_train):
dists[i][j] = np.sqrt(np.sum(np.square(self.X_train[j,:] - X[i,:])))
return dists
def compute_distances_one_loop(self, X):
num_test = X.shape[0]
num_train = self.X_train.shape[0]
dists = np.zeros((num_test, num_train))
for i in range(num_test): #i就是类别，i=1即类1
dists[i,:] = np.sqrt(np.sum(np.square(self.X_train-X[i,:]),axis = 1))
return dists
def compute_distances_no_loops(self, X):
num_test = X.shape[0]
num_train = self.X_train.shape[0]
dists = np.zeros((num_test, num_train)) #dists(测试类别数，测试集的数量)
dists = np.multiply(np.dot(X,self.X_train.T),-2)
sq1 = np.sum(np.square(X),axis=1,keepdims = True)
sq2 = np.sum(np.square(self.X_train),axis=1)
dists = np.add(dists,sq1)
dists = np.add(dists,sq2)
dists = np.sqrt(dists)
return dists
再上代码
import numpy as np
class KNearestNeighbor(object):#首先是定义一个处理KNN的类
""" a kNN classifier with L2 distance """
def __init__(self):
pass
def train(self, X, y):
self.X_train = X
self.y_train = y
def predict(self, X, k=1, num_loops=0):
if num_loops == 0:
dists = self.compute_distances_no_loops(X)
elif num_loops == 1:
dists = self.compute_distances_one_loop(X)
elif num_loops == 2:
dists = self.compute_distances_two_loops(X)
else:
raise ValueError('Invalid value %d for num_loops' % num_loops)
return self.predict_labels(dists, k=k)
def compute_distances_two_loops(self, X):
num_test = X.shape[0]
num_train = self.X_train.shape[0]
dists = np.zeros((num_test, num_train))
for i in xrange(num_test):
for j in xrange(num_train):
dists[i][j] = np.sqrt(np.sum(np.square(self.X_train[j,:] - X[i,:])))
return dists
def compute_distances_one_loop(self, X):
num_test = X.shape[0]
num_train = self.X_train.shape[0]
dists = np.zeros((num_test, num_train))
for i in range(num_test): #i就是类别，i=1即类1
dists[i,:] = np.sqrt(np.sum(np.square(self.X_train-X[i,:]),axis = 1))
return dists
def compute_distances_no_loops(self, X):
num_test = X.shape[0]
num_train = self.X_train.shape[0]
dists = np.zeros((num_test, num_train)) #dists(测试类别数，测试集的数量)
dists = np.multiply(np.dot(X,self.X_train.T),-2)
sq1 = np.sum(np.square(X),axis=1,keepdims = True)
sq2 = np.sum(np.square(self.X_train),axis=1)
dists = np.add(dists,sq1)
dists = np.add(dists,sq2)
dists = np.sqrt(dists)
return dists
def predict_labels(self, dists, k=1):
num_test = dists.shape[0]
y_pred = np.zeros(num_test)
for i in range(num_test):
closest_y = []
closest_y = self.y_train[np.argsort(dists[i,:])[:k]]
y_pred[i]=np.argmax(np.bincount(closest_y))
return y_pred
num_test = dists.shape[0]
y_pred = np.zeros(num_test)
for i in range(num_test):
closest_y = []
closest_y = self.y_train[np.argsort(dists[i,:])[:k]]
y_pred[i]=np.argmax(np.bincount(closest_y))
return y_pred
代码总结
python自己基本上算是没学过，靠着C++的底子强行直接上了，给自己留一点提示吧
def __init__(self):
pass #一般不pass，一般def __init__(self,X,y):self.X=X，就是把外部传入的变量变成类内部的东西，当然也可以像代码中再搞个方法来做这个步骤
python自己基本上算是没学过，靠着C++的底子强行直接上了，给自己留一点提示吧
def __init__(self):
pass #一般不pass，一般def __init__(self,X,y):self.X=X，就是把外部传入的变量变成类内部的东西，当然也可以像代码中再搞个方法来做这个步骤
def train(self, X, y):
self.X_train = X
self.y_train = y
这段代码就相当于做了刚才的工作
if num_loops == 0:
dists = self.compute_distances_no_loops(X)
elif num_loops == 1:
dists = self.compute_distances_one_loop(X)
elif num_loops == 2:
dists = self.compute_distances_two_loops(X)
这就是三种计算测试集中每张图片和训练集中每张图片的距离的方法，循环次数不同。
self.X_train = X
self.y_train = y
这段代码就相当于做了刚才的工作
if num_loops == 0:
dists = self.compute_distances_no_loops(X)
elif num_loops == 1:
dists = self.compute_distances_one_loop(X)
elif num_loops == 2:
dists = self.compute_distances_two_loops(X)
这就是三种计算测试集中每张图片和训练集中每张图片的距离的方法，循环次数不同。
计算之后返回dists
如果是2*2的图片的话，dists=num_test_picture*(num_train_picture个test_picture与train_picture之间的距离)
np.argsort(dists[i,:])对每一种num_test_picture对应的num_train_picture个距离进行排序
closest_y = self.y_train[np.argsort(dists[i,:])[:k]] 然后对每一种num_test_picture取前k个到closest_y，closest_y
p.argmax:最大值的索引
x = np.array([0, 1, 1, 3, 2, 1, 7]) 索引0出现了1次，索引1出现了3次......索引5出现了0次...... np.bincount(x)，因此，输出结果为：array([1, 3,1, 1, 0, 0, 0, 1])
y_pred[i] = np.argmax(np.bincount(closest_y))#就把索引出现最多的那个所以赋值到y_pred[i]
第二部分 导入数据集和做预测
# -*- coding: utf-8 -*-
import pickle as p
import numpy as np
import os
import KNearestNeighbor
def load_CIFAR_batch(filename):
""" 载入cifar数据集的一个batch """
with open(filename, 'rb') as f:
datadict = p.load(f, encoding='latin1')
X = datadict['data']
Y = datadict['labels']
X = X.reshape(10000, 3, 32, 32).transpose(0, 2, 3, 1).astype("float")
Y = np.array(Y)
return X, Y
def load_CIFAR10(ROOT):
""" 载入cifar全部数据 """
xs = []
ys = []
for b in range(1, 2):
f = os.path.join(ROOT, 'data_batch_%d' % (b,))
X, Y = load_CIFAR_batch(f)
xs.append(X) #将所有batch整合起来
ys.append(Y)
Xtr = np.concatenate(xs) #使变成行向量,最终Xtr的尺寸为(50000,32*32*3)
Ytr = np.concatenate(ys)
del X, Y
Xte, Yte = load_CIFAR_batch(os.path.join(ROOT, 'test_batch'))
return Xtr, Ytr, Xte, Yte
import numpy as np
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = (10.0, 8.0)
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'
""" 载入cifar全部数据 """
xs = []
ys = []
for b in range(1, 2):
f = os.path.join(ROOT, 'data_batch_%d' % (b,))
X, Y = load_CIFAR_batch(f)
xs.append(X) #将所有batch整合起来
ys.append(Y)
Xtr = np.concatenate(xs) #使变成行向量,最终Xtr的尺寸为(50000,32*32*3)
Ytr = np.concatenate(ys)
del X, Y
Xte, Yte = load_CIFAR_batch(os.path.join(ROOT, 'test_batch'))
return Xtr, Ytr, Xte, Yte
import numpy as np
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = (10.0, 8.0)
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'
# 载入CIFAR-10数据集
cifar10_dir = 'julyedu/datasets/cifar-10-batches-py'
X_train, y_train, X_test, y_test = load_CIFAR10(cifar10_dir)
这样算是导入了
X_train=np.reshape(X_train,(X_train.shape[0],-1))
X_test=np.reshape(X_test,(X_test.shape[0],-1))
把X_train变成行为训练集图片张数，列为单张图片的各个像素点的各个通道，如果X_train有500张图，32*32大小，彩色的图像，那X_train就变成了500*(32*32*3)的矩阵
X_train=X_train[0:10000,:]
X_test=X_test[0:500,:]
y_train=y_train[0:10000]
y_test=y_test[0:500]
然后训练集10000张，测试集取500来测试
得把之前写的KNN算法的代码导入一下，KNN算法也是我写成了KNearestNeighbor.py，那就import KNearestNeighbor，导入完就可以用算法来做预测了，分别取k为不同的数，看看预测的准确率
for k in [1, 3, 5, 10, 20, 50, 100]:
# use a particular value of k and evaluation on validation data
nn = KNearestNeighbor.KNearestNeighbor() #这里有两个KNearestNeighbor，一个是文件，后面的是文件里的类
nn.train(X_train, y_train)
# here we assume a modified NearestNeighbor class that can take a k as input
Yval_predict = nn.predict(X_test, k = k)
acc = np.mean(Yval_predict == y_test)
print( 'accuracy: %f' % (acc,))
贴一下预测结果
accuracy: 0.276000
accuracy: 0.304000
accuracy: 0.324000
accuracy: 0.306000
accuracy: 0.298000
accuracy: 0.264000
accuracy: 0.262000
结果显示k=5的时候，准确率最高，至此为止，第一步算是成功的跨出去了，花了我好几天时间，，主要是python的api都不太熟悉。