Tensorflow 2.0 学习(chapter 2)

文章目录

数据集图片分类
绘制图像
Sequential模型
Flatten层
Dense层
Model.fit()
history
归一化
Callbacks
BatchNormalization层
损失函数之: SparseCategoricalCrossentropy
Model.evaluate()
Dropout
selu
数据集房价预测
函数式API

数据集图片分类

from tensorflow import keras

fashion_mnist = keras.datasets.fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()

数据集是60000张图片及其标注, 所以其维度分别是60000×28×28, 60000×1, …

绘制图像

import matplotlib.pyplot as plt
%matplotlib inline

plt.imshow(img_arr, cmap='binary')
plt.show()

直接输入一个28×28的类型为unit8的numpy数组即可绘制图像. unit8的取值范围当然是0~255.

Sequential模型

model = keras.models.Sequential()
model.add(some layers)
model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(),
             optimizer=keras.optimizers.SGD(learning_rate=1e-3),
             metrics=['accuracy'])
model.summary()
# compile 的含义为: Configures the model for training.

Flatten层

model.add(keras.layers.Flatten(input_shape=[28, 28]))

input_shape: Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

继承自基类 tf.keras.layers.Layer
data_format: A string, one of channels_last (default) or channels_first.
The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, ..., channels) while channels_first corresponds to inputs with shape (batch, channels, ...). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be “channels_last”.

Dense层

model.add(keras.layers.Dense(300, activation='relu'))

Dense层有很多参数, 官方文档中介绍, 默认参数如下:

tf.keras.layers.Dense(
    units, activation=None, use_bias=True, kernel_initializer='glorot_uniform',
    bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None,
    activity_regularizer=None, kernel_constraint=None, bias_constraint=None,
    **kwargs
)

其中kernel和bias分别是权重和偏置, wx+b中的w和b.

从参数中可以看到, 可以设置初始值和正则化.

Model.fit()

history = model.fit(x_train, y_train, epochs=10, validation_data=(x_valid, y_valid))

详细信息参见: https://tensorflow.google.cn/api_docs/python/tf/keras/Model#fit

支持的参数包括: 输入(分别指定x和y或者当x是Dataset时只指定x), batch_size(默认为32), epochs, 进度条格式(无,整体进度条,每个epoch一条), 验证集分割, 独立的验证集, 验证频率等

history

history.history是dict{str–>list}类型, 转换为DataFrame后即可绘制.

Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

归一化

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
x_train_scaled = scaler.fit_transform(
    x_train.astype(np.float32).reshape(-1, 1)).reshape(-1, 28, 28)
x_valid_scaled = scaler.transform(
    x_valid.astype(np.float32).reshape(-1, 1)).reshape(-1, 28, 28)
x_test_scaled = scaler.transform(
    x_test.astype(np.float32).reshape(-1, 1)).reshape(-1, 28, 28)

使用numpy数组的astype函数将类型转换为float32类型.
使用numpy.reshape()函数将形状转换为 43120000*1 的数组.

ndarray.reshape(shape,order='C')
用StandardScalar.fit_transform方法进行归一化

``StandardScaler` 标准缩放器

The standard score of a sample x is calculated as:

z = (x - u) / s

Callbacks

logdir = os.path.join('.', 'callbacks')
if os.path.exists(logdir) == False: os.mkdir(logdir)
output_model_file = os.path.join(logdir, "fashion_mnist_model.h5")
callbacks = [
	keras.callbacks.TensorBoard(logdir),
    keras.callbacks.ModelCheckpoint(output_model_file, save_best_only=True),
    keras.callbacks.EarlyStopping(patience=5, min_delta=1e-3),]
history = model.fit(..., callbacks=callbacks)

在fit时执行的回调. 常用的Callback就是上面三个: TensorBoard、Checkpoint、EarlyStopping.

BatchNormalization层

# 批归一化放在激活函数之前
model.add(keras.layers.BatchNormalization())
# 批归一化放在激活函数之后
model.add(keras.layers.Dense(100))
model.add(keras.layers.BatchNormalization()) # 批归一化放在激活函数之前
model.add(keras.layers.Activation('relu'))   # 激活函数是单独的一层

Normalize and scale inputs or activations. (Ioffe and Szegedy, 2014).

Normalize the activations of the previous layer at each batch, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1.

归一化及缩放输入或激活函数.

损失函数之: SparseCategoricalCrossentropy

model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(), ..)

Computes the crossentropy loss between the labels and predictions.

类别标签和 one-hot 风格的概率分布，比如 y=1, ypred=[0.05,0.89,0.06]

Model.evaluate()

evaluate(
    x=None, y=None, batch_size=None, verbose=1, sample_weight=None, steps=None,
    callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False
)

Returns the loss value & metrics values for the model in test mode.

Computation is done in batches.

Dropout

# alphaDropout保证丢弃之后的均值和方差不变; 
# 归一化性质不变, 这样的话可以跟归一化操作一块使用, 不影响.
model.add(keras.layers.AlphaDropout(rate=0.5))   # 一般dropout是0.5;
model.add(keras.layers.Dropout(rate=0.5))   # 普通的dropout

selu

自带归一化的激活函数

数据集房价预测

from sklearn.datasets import fetch_california_housing
housing = fetch_california_housing()

from sklearn.model_selection import train_test_split
x_train_all, x_test, y_train_all, y_test = 
	train_test_split(housing.data, housing.target, random_state=7)
x_train, x_valid, y_train, y_valid = 
	train_test_split(x_train_all, y_train_all, random_state=11)

housing.data是20640×8的np数组, housing.target是20640的一维np数组
sklearn.model_selection.train_test_split

Split arrays or matrices into random train and test subsets

Quick utility that wraps input validation and next(ShuffleSplit().split(X, y)) and application to input data into a single call for splitting (and optionally subsampling) data in a oneliner.

默认3:1, 可以设置train_size或test_size进行修改, 参考官方文档

函数式API

包括tf.keras.layers中的函数

Input(...): Input() is used to instantiate a Keras tensor.

concatenate(...): Functional interface to the Concatenate layer.

add(...): Functional interface to the Add layer.

和tf.keras.layers.Layers.__call__()方法

InputLayer(...): Layer to be used as an entry point into a Network (a graph of layers).

以及其它的Layer等.

知识点: 如果python类定义了__call__方法，那么它的实例可以作为函数进行调用。

input = keras.layers.Input(shape=x_train.shape[1:])
hidden = keras.layers.Dense(30, activation='relu')(input)
concat = keras.layers.concatenate([input, hidden]) # 连接两个层
output = keras.layers.Dense(1)(concat)
# 在函数式API下, 指定图的输入和输出, 构造Model
model = keras.models.Model(inputs=[input], outputs=[output])

自定义Model

class WideDeepModel(keras.models.Model):
    def __init__(self):
        super(WideDeepModel, self).__init__()
        """定义模型层次"""
        self.hidden1_layer = keras.layers.Dense(30, activation="relu")
        self.hidden2_layer = keras.layers.Dense(30, activation="relu")
        self.output_layer = keras.layers.Dense(1)
        
    def call(self, input):
        """完成模型的正向计算"""
        hidden1 = self.hidden1_layer(input)
        hidden2 = self.hidden2_layer(hidden1)
        concat = keras.layers.concatenate([input, hidden2])
        output = self.output_layer(concat)
        return output
    
model = WideDeepModel()
# or: model = keras.models.Sequential([WideDeepModel()])
model.build(input_shape=(None, 8))

super的功能

Python3.x 和 Python2.x 的一个区别是: Python 3 可以使用直接使用 super().xxx 代替 super(Class, self).xxx
实现方式

首先继承keras.models.Model类, 然后在构造函数中创建3个层次, 然后在call方法中用函数式API计算.

多输入

在构建Model的时候可以指定多个输入层, 同样是基于函数式API.

input_wide = keras.layers.Input(shape=[5])
input_deep = keras.layers.Input(shape=[6])
concat = keras.layers.concatenate([input_wide, input_deep])
output = keras.layers.Dense(1)(concat)
model = keras.models.Model(inputs=[input_wide, input_deep], outputs=[output])

两个输入可以来自两个数组, 也可以在一个数组上取不同的维度, 但是注意对齐.

参数搜索

from scipy.stats import reciprocal
from sklearn.model_selection import RandomizedSearchCV
param_distribution = {
    "hidden_layers": [1, 2, 3, 4],
    "layer_size": np.arange(1, 100),
    "learning_rate": reciprocal(1e-4, 1e-2)
}

def build_model(hidden_layers=1, layer_size=30, learning_rate=3e-3):
    model = keras.models.Sequential()
    model.add(some layers)
    model.compile(loss='mse', optimizer=keras.optimizers.SGD(learning_rate))
    return model

model = keras.wrappers.scikit_learn.KerasRegressor(build_model)
rsc = RandomizedSearchCV(model, param_distribution, n_iter=10, n_jobs=1)
rsc.fit(x_train_scaled, y_train, epochs=100, ...) # 与模型的fit一样

整体过程

首先定义多个参数及参数值范围, 然后定义一个函数返回keras.Model, 然后经过scikit_learn.KerasRegressor及sklearn.model_selection.RandomizedSearchCV两层封装, 最后fit.
scipy.stats.reciprocal

A reciprocal continuous random variable.

倒数连续随机变量. 注意是一个变量.
KerasRegressor

把keras Model封装成scikit Regressor, 然后用scikit的相关API进行操作.

Implementation of the scikit-learn regressor API for Keras.
```
tf.keras.wrappers.scikit_learn.KerasRegressor(build_fn=None, **sk_params)
```
sk_params 还可以接受用于调用 fit，predict，predict_proba 和 score 方法的参数（例如，epochs，batch_size）。训练（预测）参数按以下顺序选择：
1. 传递给 fit，predict，predict_proba 和 score 函数的字典参数的值；
2. 传递给 sk_params 的值；
3. keras.models.Sequential 的 fit，predict，predict_proba 和 score 方法的默认值。
详细文档参见https://keras.io/zh/scikit-learn-api/
RandomizedSearchCV

Randomized search on hyper parameters. 一种超参数搜索算法.

RandomizedSearchCV implements a “fit” and a “score” method. It also implements “predict”, “predict_proba”, “decision_function”, “transform” and “inverse_transform” if they are implemented in the estimator used.

The parameters of the estimator used to apply these methods are optimized by cross-validated search over parameter settings.

In contrast to GridSearchCV, not all parameter values are tried out, but rather a fixed number of parameter settings is sampled from the specified distributions. The number of parameter settings that are tried is given by n_iter.

If all parameters are presented as a list, sampling without replacement is performed. If at least one parameter is given as a distribution, sampling with replacement is used. It is highly recommended to use continuous distributions for continuous parameters.

Auliegay

发布了80 篇原创文章 · 获赞 22 · 访问量 5万+

私信关注