Keras【Deep Learning With Python】MNIST数据集识别优化

前言

本文分为三部分：

a.线性回归
b.手写数字识别
c.手写数字识别模型优化。

1 线性回归预测

import keras

Using TensorFlow backend.
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:521: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:522: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

x=np.linspace(0,100,30)
y=3*x+7+np.random.randn(30)*6

x

array([  0.        ,   3.44827586,   6.89655172,  10.34482759,
        13.79310345,  17.24137931,  20.68965517,  24.13793103,
        27.5862069 ,  31.03448276,  34.48275862,  37.93103448,
        41.37931034,  44.82758621,  48.27586207,  51.72413793,
        55.17241379,  58.62068966,  62.06896552,  65.51724138,
        68.96551724,  72.4137931 ,  75.86206897,  79.31034483,
        82.75862069,  86.20689655,  89.65517241,  93.10344828,
        96.55172414, 100.        ])

y

array([  4.51440317,  26.25914239,  31.31959183,  40.88305373,
        47.23353782,  52.12131196,  63.47049378,  69.38831163,
        91.34933604, 100.32051013, 106.63513579, 121.44319809,
       143.85751839, 133.16091292, 157.33693677, 167.16364628,
       169.22570758, 175.27839772, 196.90560814, 212.32010811,
       215.80391853, 223.70326853, 233.37777019, 241.45304151,
       255.68397824, 266.73365369, 276.66691758, 281.3290158 ,
       295.98657774, 302.39809267])

plt.scatter(x,y)

y=ax+b

#建立模型
model=keras.Sequential() # 顺序模型

from keras import layers

#添加层 指定输入输出维度 初始化两个param（a，b） 之后训练ab
model.add(layers.Dense(1,input_dim=1))

model.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 1)                 2         
=================================================================
Total params: 2
Trainable params: 2
Non-trainable params: 0
_________________________________________________________________

#编译模型 loss是什么 使用什么算法优化loss

model.compile(optimizer='adam',
              loss='mse'
)

#训练模型

model.fit(x,y,epochs=3000)

Epoch 1/3000
30/30 [==============================] - 0s 13ms/step - loss: 13768.0615
..........................................................................

Epoch 2568/3000
30/30 [==============================] - 0s 67us/step - loss: 248.5746

plt.scatter(x,y,c='r')
plt.plot(x,model.predict(x))

model.predict([150])

array([[442.45358]], dtype=float32)

#442

2 手写数字识别

import keras
from keras import layers
import matplotlib.pyplot as plt
%matplotlib inline

Using TensorFlow backend.
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:521: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:522: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
c:\program files\python36\lib\site-packages\tensorflow\python\framework\dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])

import keras.datasets.mnist as minst

(train_image,train_label),(test_image,test_label)=minst.load_data()

train_image.shape

(60000, 28, 28)

plt.imshow(train_image[0])

train_label[0]

5

train_label

array([5, 0, 4, ..., 5, 6, 8], dtype=uint8)

test_image.shape

(10000, 28, 28)

model=keras.Sequential()
model.add(layers.Flatten(input_shape=[28,28])) #展平 （60000，28，28）-》（60000，28*28）
model.add(layers.Dense(64,activation='relu'))# 关注第二维度 输入28*28 全连接64个单元 再输出到10个单元
model.add(layers.Dense(10,activation='softmax'))# 多分类 使用softmax激活

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',#交叉熵
              metrics=['acc']
)

model.fit(train_image,train_label,epochs=50,batch_size=512)

Epoch 1/50
60000/60000 [==============================] - 1s 20us/step - loss: 8.0500 - acc: 0.4950

Epoch 50/50
60000/60000 [==============================] - 1s 11us/step - loss: 1.7518 - acc: 0.8901





<keras.callbacks.History at 0x199f81df908>

model.evaluate(test_image, test_label)#模型评价

10000/10000 [==============================] - 0s 45us/step





[1.8363827221155036, 0.8847]

model.evaluate(train_image, train_label)#该模型在训练集和测试集得分差别不大，得分都比较低，属于欠拟合

60000/60000 [==============================] - 2s 34us/step





[1.7373228628097839, 0.8907666666666667]

import numpy as np
np.argmax(model.predict(test_image[:10]), axis=1)#axis=1 在第一维，最后一维上

array([7, 2, 1, 0, 4, 1, 4, 9, 6, 9], dtype=int64)

test_label[:10]

array([7, 2, 1, 0, 4, 1, 4, 9, 5, 9], dtype=uint8)

#对比可得预测错误一个

3 模型优化

增大网络容量，直至过拟合
增加两个隐藏层

model = keras.Sequential()
model.add(layers.Flatten())     # (60000, 28, 28) --->  (60000, 28*28)
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10, activation='softmax'))

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['acc']
)
model.fit(train_image, train_label, epochs=50, batch_size=512, validation_data=(test_image, test_label))

训练集测试集正确率得分都很高
增大容量可以有效提高准确度
分析测试正确率和训练正确率，我们发现网络还是有一些过拟合，所以采用Dropout抑制过拟合

model = keras.Sequential()
model.add(layers.Flatten())     # (60000, 28, 28) --->  (60000, 28*28)
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(10, activation='softmax'))

增加Dropout层则训练模型时epoch增加些，让其多训练几次

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['acc']
)
model.fit(train_image, train_label, epochs=200, batch_size=512, validation_data=(test_image, test_label))

分析数据，训练数据得分下降一些，限额是数据得分不变。抑制了过拟合。