最近在学习TensorFlow，学习了使用TensorFlow来实现LSTM（长短时记忆），并学习了LSTM在分类方面的示例。
即应用TensorFlow实现LSTM，对 MNIST 数据集进行分类。
下面是示例代码：
#encoding:utf-8
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
# 导入数据
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
# hyperparameters 超参数
lr = 0.001
training_iters = 100000
batch_size = 128
n_inputs = 28 # (img shape:28 * 28)
n_steps = 28 # time steps
n_hidden_units = 128 # 隐藏层神经元数目
n_classes = 10 # classes(0-9 digits)
# 输入
x = tf.placeholder(tf.float32, [None, n_steps, n_inputs])
y = tf.placeholder(tf.float32, [None, n_classes])
# 定义权值
weights = {
# (28, 128)
'in': tf.Variable(tf.random_normal([n_inputs, n_hidden_units])),
# (128, 10)
'out': tf.Variable(tf.random_normal([n_hidden_units, n_classes]))
}
biases = {
# (128, )
'in': tf.Variable(tf.constant(0.1, shape=[n_hidden_units, ])),
# (10, )
'out': tf.Variable(tf.constant(0.1, shape=[n_classes]))
}
def RNN(X, weights, biases):
# 隐藏层输入到cell
# X(128 batch, 28 steps, 28 inputs)
# ==>(128*28,28 inputs)
X = tf.reshape(X, [-1, n_inputs])
# X_in==>(128batch*28steps, 128 hidden)
X_in = tf.matmul(X, weights['in'] + biases['in'])
# X_in==>(128batch, 28steps, 128 hidden)
X_in = tf.reshape(X_in, [-1, n_steps, n_hidden_units])
# cell
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(n_hidden_units, forget_bias=1.0, state_is_tuple=True)
# lstm cell 被分成两部分，(c_state, m_state)
_init_state = lstm_cell.zero_state(batch_size, dtype=tf.float32)
outputs, states = tf.nn.dynamic_rnn(lstm_cell, X_in, initial_state=_init_state, time_major=False)
# 隐藏层输出
# 第一种方式
results = tf.matmul(states[1], weights['out']) + biases['out']
# 第二种方式
# outputs = tf.unstack(tf.transpose(outputs, [1, 0, 2])) # state is the last outputs
# results = tf.matmul(outputs[-1], weights['out']) + biases['out']
return results
pred = RNN(x, weights, biases)
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))
train_op = tf.train.AdamOptimizer(lr).minimize(cost)
correct_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
step = 0
while step * batch_size < training_iters:
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
batch_xs = batch_xs.reshape([batch_size, n_steps, n_inputs])
sess.run([train_op], feed_dict={
x: batch_xs,
y: batch_ys,
})
if step % 20 == 0:
print(sess.run(accuracy, feed_dict={
x: batch_xs,
y: batch_ys,
}))
step += 1运行结果：
Extracting MNIST_data\train-images-idx3-ubyte.gz
Extracting MNIST_data\train-labels-idx1-ubyte.gz
Extracting MNIST_data\t10k-images-idx3-ubyte.gz
Extracting MNIST_data\t10k-labels-idx1-ubyte.gz
WARNING:tensorflow:From D:\Users\Seavan_CC\Anaconda3\lib\site-packages\tensorflow\python\util\tf_should_use.py:170: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
2018-07-16 15:37:04.257479: W c:\l\tensorflow_1501918863922\work\tensorflow-1.2.1\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE instructions, but these are available on your machine and could speed up CPU computations.
2018-07-16 15:37:04.257851: W c:\l\tensorflow_1501918863922\work\tensorflow-1.2.1\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE2 instructions, but these are available on your machine and could speed up CPU computations.
2018-07-16 15:37:04.258213: W c:\l\tensorflow_1501918863922\work\tensorflow-1.2.1\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
2018-07-16 15:37:04.258573: W c:\l\tensorflow_1501918863922\work\tensorflow-1.2.1\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-07-16 15:37:04.258934: W c:\l\tensorflow_1501918863922\work\tensorflow-1.2.1\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-07-16 15:37:04.259296: W c:\l\tensorflow_1501918863922\work\tensorflow-1.2.1\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-07-16 15:37:04.259639: W c:\l\tensorflow_1501918863922\work\tensorflow-1.2.1\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-07-16 15:37:04.259977: W c:\l\tensorflow_1501918863922\work\tensorflow-1.2.1\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
0.132813
0.679688
0.703125
0.820313
0.851563
0.867188
0.828125
0.898438
0.898438
0.921875
0.898438
0.84375
0.898438
0.921875
0.90625
0.9375
0.945313
0.9375
0.960938
0.929688
0.929688
0.929688
0.9375
0.984375
0.960938
0.976563
0.96875
0.960938
0.96875
0.960938
0.921875
0.945313
0.96875
0.953125
0.960938
0.960938
0.984375
0.976563
0.976563
0.960938注：MNIST数据集的下载问题可参考我另一篇博客：TensorFlow MNIST数据集手写数字识别（并解决MNIST数据集下载问题）