TensorFlow 1.8 learning record

1. Installation

cpu version:

conda install tensorflow==1.8 -i https://mirrors.aliyun.com/pypi/simple

https://www.cnblogs.com/liuhuacai/p/11684666.html
install cudatoolkit10.0.130 cudnn7.6.5
update numpy: install numpy==1.16.0

2. Framework introduction

2.1 TF data flow diagram

Construction diagram stage:

Figure: tensorflow expresses calculation as a dependency between instructions, which
can be understood as a flowchart, which defines data (tensor Tensor) and operation (node ​​Op)

Execution diagram stage

Session: A mechanism
for running data flow graphs across one or more local or remote devices to allow previously defined data and operations to run

Node: Provides operations performed in the graph

2.2 Graphs and TensorBoard

2.2.1 What is a graph structure

Graph structure: data (Tensor) + operation (Operation)

2.2.2 Diagram related operations

1. Default map

Usually TensorFlow will help create a graph by default

1. Call method

tf.get_default_graph()

2. View properties

.gragh

2. Create a graph

new_g = tf.graph()
with new_g.as_default():
	定义数据和操作

2.2.3 TensorBoard: visual learning

1. Data serialization-events file

Serialize the graph data and write it to the file

tf.summary.FileWriter(path, graph=sess.graph)

2. Start TensorBoard

tensorboard --logdir="./tmp/summary"

127.0.0.1:6006

2.2.4 OP

Data: Tensor Object
Operation: Operation Object-OP

1 Common OP

Operation function	Operation object
tf.constant (Tensor object)	Input Tensor Object-Const-Output Tensor Object
tf.add(Tensor object 1, Tensor object 2)	Input Tensor Object 1, Tensor Object 2-Add Object-Output Tensor Object 3

In the process of calling the operation function,
an operation object will be generated. An operation object (Operation) is a node in the TensorFlow graph. It can accept 0 or more input Tensors, and can output 0 or more Tensors. The Operation object is passed through op Constructor (such as tf.matmul()) created.

2 Command name

One picture, one namespace

2.3 Session

2.3.1 Session

tf.Session: used in complete programs
tf.InteractiveSession: used in TensorFlow in interactive contexts, such as shell

1 init(target=’’, graph=None, config=None)

A session is an object that owns resources, such as tf.Variable, tf.QueueBase, and tf.ReaderBase. When these resources are no longer needed, they must be released. tf.Session.close Use context manager
target: Used to specify the address of the TensorFlow server
config: Use tf.ConfigProto to print device usage information (on which device the operation is running)

2 session run()

run(fetches,feed_dict=None,options=None,run_metadata=None)
fetches: single operation, or list, ancestor (other types that do not belong to tensorflow will not work)

a = tf.constant(5.0)
b = tf,cibstabt(6.0)
c = a * b

# 创建会话
sess = tf.Session()

# 计算C的值
print(sess.run(c))
print(c.eval(session=sess))

feed_dict: Used in conjunction with tf.placeholder to check whether the value shape is compatible with the placeholder . The placeholder provides placeholders, and the parameters are specified by feed_dict when running.
Usage scenario: When defining the tensor, the specific value is not determined (it can be understood as a variable of the tensor type)

Runtime error error:
RuntimeError: Session是无效状态（如已关闭）
TypeError: fetches或者feed_dict键的类型不合适
ValueError: fetches或feed_dict键无效或引用Tensor不存在的键。

2.4 Tensor

TensorFlow's tensor is an n-dimensional array of type tf.Tensor. There are two important attributes: type: data type, shape: shape (order)

2.4.1 Tensor

Scalar	A number
vector	One-dimensional array [2,3,4]
matrix	Two-dimensional array [[2,3,4],[2,3,4]]
。。。。	。。。。
Tensor	n-dimensional array

1 tensor type

2 tensor order

2.4.2 Instructions for creating tensors

Fixed value tensor

tf.zeros(shape, dtype=tf.float32, name=None)
tf.ones(shape, dtype=tf.float32, name=None)
tf.constant(shape, dtype=tf.float32, name=None)
# tf.ones(shape=[3, 4])
.eval()

####Random value tensor

tf.random_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32)
# mean:均值   stddev:标准差

2.4.3 Transformation of tensors

Modification of
ndarray attributes: Modification of type: 1) ndarray.astype(type) 2) Modification of ndarray.tostring()
shape: 1) ndarray.reshape(shape) -1 automatically calculates the shape and returns a new array 2) ndarray.resize( shape) directly modify the original array

1 Type change

tf.cast(tensor, dtype, name=None)
does not change the original tensor, and returns the new tensor after the changed type

2 Shape change

Static shape-the shape when the tensor is created
1) How to change the static shape The static shape
can be changed and updated only when the shape is not completely fixed
tensor.set_shape(shape)
2) How to change the dynamic shape (can change the order)
tf.reshape(tensor, shape)
will not change the original tensor, and return the new tensor after the changed type
must ensure that the number of elements of the tensor must match (for example: 2, 3 is changed to 3, 2)

2.4.4 Mathematical operations on tensors

Arithmetic operators
Basic mathematical functions
Matrix operations
reduce operations
Sequence index operations

2.5 Variable OP

Unique in TensorFlow-Variables, features: storage persistence, modifiable values, can be specified to be trained,
can be understood as designed for storing model parameters

2.5.1 Create Variable

tf.Variable(initial_value=None, trainable=True, collections=None,name=None)
initial_value: initialized value
trainable: whether to be trained
Note: Variables need to be explicitly initialized before they can run values

2.5.2 Use tf.variable_scope() to modify the namespace of variables

Make the structure clearer

2.6 Advanced API

2.6.1 Basic API

1 tf.app

Provide a main function entry for the script run by TensorFlow

2 tf.image

3 tf.gfile

4 tf.summary

It is used to generate statistical logs available to TensorFlow. Currently, it mainly provides 4 types: audio, image, histogram, scalar

2 tf.python_io

2 tf.train

2 tf.nn

Provides some low-level functions for building neural networks. TensorFlow builds the core module of the network. It contains functions for adding various layers, such as adding convolutional layers, pooling layers, and so on.

2.6.2 Advanced API

2 tf. Hard

2 tf.layers

2 tf.contrib

2 tf.estimator

2.7 Case: Linear regression

2.7.1 Principle

1) Build the model
2) Construct the mean square error of the loss function
3) Optimize the loss gradient descent

2.7.2 Case realization

1 Case determination

100 samples

2 API

Matrix multiplication tf.matmul(x, w)
square tf.square(error)
mean tf.reduce_mean(eooro)
gradient descent optimization tf.train.GradientDescentOptimizer(learning_rate)

5 Setting of learning rate, setting of steps and gradient explosion

Gradient explosion: In extreme cases, the value of the weight becomes so large that it overflows, resulting in the NaN value

6 Observation of variable trainable settings

Specify whether the variable is trained

2.7.3 Add other functions

1 TensorBoard adds variable display

1. Collect variables
tf.summary.scalar(name="",tensor) scalar
tf.summary.histogram(name='',tensor) collect high-dimensional variable parameters
tf.summary.image(name='',tensor) Picture
2. The merge variable is written into the event file
merged = tf.summary.merge_all()
Run merge: summary=sess.run(merged), each iteration needs to be run
Add: FileWriter.add_summary(summary, i) i represents the number Second value

1. Create event file
2. Collect variables
3. Merge variables
4. Run merge variables once per iteration
5. Write the summary object to the event file for each iteration

2 Increase the namespace

3 Save and load the model

tf.train.Saver(var_list=None, max_to_keep=5)
var_list: Specify the variables to be saved and restored.
max_to_keep: Indicates the maximum number of recent checkpoint files to be kept.
1) Instantiate Saver
2) Save
saver.save(sess, path)
3) Load
saver.restore(sess, path)

4 Use of command line parameters

1. tf.app.flags
supports the application to accept parameters from the command line, which can be used to specify the cluster configuration, etc.
tf.app.flags.DEFINE_integer(“max_step”, 0, “number of steps to train the model”) Parameter name, default value, variable description
tf.app.flags.DEFINE_string(“model_dir”, “”, “The path where the model is saved +Model name")
2. FLAGS = tf.app.flags.FLAGS
calls the parameters passed in the command line
through FLAGS.max_step 3. Starts the main(argv) function through tf.app.run()