本文在调参记录6的基础上，继续调整超参数，测试Adaptively Parametric ReLU（APReLU）激活函数在Cifar10图像集上的效果。
深度残差网络+自适应参数化ReLU激活函数（调参记录6）
https://www.cnblogs.com/shisuzanian/p/12907482.html
自适应参数化ReLU激活函数的基本原理见下图：
在Keras里，Batch Normalization的momentum默认为0.99，现在设置为0.9，这是因为momentum=0.9似乎更常见。原先Batch Normalization默认没有正则化，现在加上L2正则化，来减小过拟合。
Keras程序如下：
1 #!/usr/bin/env python3 2 # -*- coding: utf-8 -*- 3 """ 4 Created on Tue Apr 14 04:17:45 2020 5 Implemented using TensorFlow 1.0.1 and Keras 2.2.1 6 7 Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, 8 Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, 9 IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458 10 11 @author: Minghang Zhao 12 """ 13 14 from __future__ import print_function 15 import keras 16 import numpy as np 17 from keras.datasets import cifar10 18 from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimum 19 from keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshape 20 from keras.regularizers import l2 21 from keras import backend as K 22 from keras.models import Model 23 from keras import optimizers 24 from keras.preprocessing.image import ImageDataGenerator 25 from keras.callbacks import LearningRateScheduler 26 K.set_learning_phase(1) 27 28 # The data, split between train and test sets 29 (x_train, y_train), (x_test, y_test) = cifar10.load_data() 30 31 # Noised data 32 x_train = x_train.astype('float32') / 255. 33 x_test = x_test.astype('float32') / 255. 34 x_test = x_test-np.mean(x_train) 35 x_train = x_train-np.mean(x_train) 36 print('x_train shape:', x_train.shape) 37 print(x_train.shape[0], 'train samples') 38 print(x_test.shape[0], 'test samples') 39 40 # convert class vectors to binary class matrices 41 y_train = keras.utils.to_categorical(y_train, 10) 42 y_test = keras.utils.to_categorical(y_test, 10) 43 44 # Schedule the learning rate, multiply 0.1 every 300 epoches 45 def scheduler(epoch): 46 if epoch % 300 == 0 and epoch != 0: 47 lr = K.get_value(model.optimizer.lr) 48 K.set_value(model.optimizer.lr, lr * 0.1) 49 print("lr changed to {}".format(lr * 0.1)) 50 return K.get_value(model.optimizer.lr) 51 52 # An adaptively parametric rectifier linear unit (APReLU) 53 def aprelu(inputs): 54 # get the number of channels 55 channels = inputs.get_shape().as_list()[-1] 56 # get a zero feature map 57 zeros_input = keras.layers.subtract([inputs, inputs]) 58 # get a feature map with only positive features 59 pos_input = Activation('relu')(inputs) 60 # get a feature map with only negative features 61 neg_input = Minimum()([inputs,zeros_input]) 62 # define a network to obtain the scaling coefficients 63 scales_p = GlobalAveragePooling2D()(pos_input) 64 scales_n = GlobalAveragePooling2D()(neg_input) 65 scales = Concatenate()([scales_n, scales_p]) 66 scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) 67 scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales) 68 scales = Activation('relu')(scales) 69 scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) 70 scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales) 71 scales = Activation('sigmoid')(scales) 72 scales = Reshape((1,1,channels))(scales) 73 # apply a paramtetric relu 74 neg_part = keras.layers.multiply([scales, neg_input]) 75 return keras.layers.add([pos_input, neg_part]) 76 77 # Residual Block 78 def residual_block(incoming, nb_blocks, out_channels, downsample=False, 79 downsample_strides=2): 80 81 residual = incoming 82 in_channels = incoming.get_shape().as_list()[-1] 83 84 for i in range(nb_blocks): 85 86 identity = residual 87 88 if not downsample: 89 downsample_strides = 1 90 91 residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual) 92 residual = aprelu(residual) 93 residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), 94 padding='same', kernel_initializer='he_normal', 95 kernel_regularizer=l2(1e-4))(residual) 96 97 residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual) 98 residual = aprelu(residual) 99 residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', 100 kernel_regularizer=l2(1e-4))(residual) 101 102 # Downsampling 103 if downsample_strides > 1: 104 identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity) 105 106 # Zero_padding to match channels 107 if in_channels != out_channels: 108 zeros_identity = keras.layers.subtract([identity, identity]) 109 identity = keras.layers.concatenate([identity, zeros_identity]) 110 in_channels = out_channels 111 112 residual = keras.layers.add([residual, identity]) 113 114 return residual 115 116 117 # define and train a model 118 inputs = Input(shape=(32, 32, 3)) 119 net = Conv2D(16, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs) 120 net = residual_block(net, 9, 16, downsample=False) 121 net = residual_block(net, 1, 32, downsample=True) 122 net = residual_block(net, 8, 32, downsample=False) 123 net = residual_block(net, 1, 64, downsample=True) 124 net = residual_block(net, 8, 64, downsample=False) 125 net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net) 126 net = Activation('relu')(net) 127 net = GlobalAveragePooling2D()(net) 128 outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net) 129 model = Model(inputs=inputs, outputs=outputs) 130 sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True) 131 model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy']) 132 133 # data augmentation 134 datagen = ImageDataGenerator( 135 # randomly rotate images in the range (deg 0 to 180) 136 rotation_range=30, 137 # randomly flip images 138 horizontal_flip=True, 139 # randomly shift images horizontally 140 width_shift_range=0.125, 141 # randomly shift images vertically 142 height_shift_range=0.125) 143 144 reduce_lr = LearningRateScheduler(scheduler) 145 # fit the model on the batches generated by datagen.flow(). 146 model.fit_generator(datagen.flow(x_train, y_train, batch_size=100), 147 validation_data=(x_test, y_test), epochs=1000, 148 verbose=1, callbacks=[reduce_lr], workers=4) 149 150 # get results 151 K.set_learning_phase(0) 152 DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0) 153 print('Train loss:', DRSN_train_score[0]) 154 print('Train accuracy:', DRSN_train_score[1]) 155 DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0) 156 print('Test loss:', DRSN_test_score[0]) 157 print('Test accuracy:', DRSN_test_score[1])
实验结果如下：
1 x_train shape: (50000, 32, 32, 3) 2 50000 train samples 3 10000 test samples 4 Epoch 1/1000 5 97s 195ms/step - loss: 3.2344 - acc: 0.4133 - val_loss: 2.7840 - val_acc: 0.5398 6 Epoch 2/1000 7 65s 131ms/step - loss: 2.6095 - acc: 0.5574 - val_loss: 2.3084 - val_acc: 0.6296 8 Epoch 3/1000 9 65s 131ms/step - loss: 2.2160 - acc: 0.6249 - val_loss: 1.9625 - val_acc: 0.6837 10 Epoch 4/1000 11 65s 131ms/step - loss: 1.9251 - acc: 0.6702 - val_loss: 1.7395 - val_acc: 0.7116 12 Epoch 5/1000 13 65s 131ms/step - loss: 1.7015 - acc: 0.7016 - val_loss: 1.5316 - val_acc: 0.7429 14 Epoch 6/1000 15 65s 131ms/step - loss: 1.5268 - acc: 0.7228 - val_loss: 1.3858 - val_acc: 0.7608 16 Epoch 7/1000 17 65s 131ms/step - loss: 1.3979 - acc: 0.7372 - val_loss: 1.2604 - val_acc: 0.7761 18 Epoch 8/1000 19 65s 131ms/step - loss: 1.2921 - acc: 0.7483 - val_loss: 1.1713 - val_acc: 0.7798 20 Epoch 9/1000 21 66s 131ms/step - loss: 1.2057 - acc: 0.7627 - val_loss: 1.1200 - val_acc: 0.7846 22 Epoch 10/1000 23 65s 131ms/step - loss: 1.1358 - acc: 0.7690 - val_loss: 1.0900 - val_acc: 0.7811 24 Epoch 11/1000 25 65s 131ms/step - loss: 1.0823 - acc: 0.7741 - val_loss: 0.9822 - val_acc: 0.8058 26 Epoch 12/1000 27 65s 131ms/step - loss: 1.0365 - acc: 0.7802 - val_loss: 0.9840 - val_acc: 0.7976 28 Epoch 13/1000 29 65s 130ms/step - loss: 1.0040 - acc: 0.7847 - val_loss: 0.9539 - val_acc: 0.7995 30 Epoch 14/1000 31 65s 131ms/step - loss: 0.9737 - acc: 0.7870 - val_loss: 0.9181 - val_acc: 0.8093 32 Epoch 15/1000 33 65s 131ms/step - loss: 0.9468 - acc: 0.7933 - val_loss: 0.8972 - val_acc: 0.8071 34 Epoch 16/1000 35 65s 131ms/step - loss: 0.9210 - acc: 0.7964 - val_loss: 0.9039 - val_acc: 0.8077 36 Epoch 17/1000 37 65s 131ms/step - loss: 0.9084 - acc: 0.8008 - val_loss: 0.8491 - val_acc: 0.8200 38 Epoch 18/1000 39 65s 131ms/step - loss: 0.8879 - acc: 0.8027 - val_loss: 0.8565 - val_acc: 0.8161 40 Epoch 19/1000 41 65s 131ms/step - loss: 0.8770 - acc: 0.8044 - val_loss: 0.8640 - val_acc: 0.8116 42 Epoch 20/1000 43 65s 131ms/step - loss: 0.8695 - acc: 0.8066 - val_loss: 0.8369 - val_acc: 0.8187 44 Epoch 21/1000 45 65s 131ms/step - loss: 0.8565 - acc: 0.8097 - val_loss: 0.8403 - val_acc: 0.8221 46 Epoch 22/1000 47 65s 131ms/step - loss: 0.8516 - acc: 0.8119 - val_loss: 0.8131 - val_acc: 0.8315 48 Epoch 23/1000 49 65s 131ms/step - loss: 0.8402 - acc: 0.8156 - val_loss: 0.7879 - val_acc: 0.8397 50 Epoch 24/1000 51 65s 131ms/step - loss: 0.8271 - acc: 0.8179 - val_loss: 0.7942 - val_acc: 0.8379 52 Epoch 25/1000 53 65s 131ms/step - loss: 0.8282 - acc: 0.8196 - val_loss: 0.8132 - val_acc: 0.8270 54 Epoch 26/1000 55 65s 130ms/step - loss: 0.8203 - acc: 0.8203 - val_loss: 0.7870 - val_acc: 0.8354 56 Epoch 27/1000 57 65s 131ms/step - loss: 0.8141 - acc: 0.8231 - val_loss: 0.7780 - val_acc: 0.8405 58 Epoch 28/1000 59 65s 131ms/step - loss: 0.8075 - acc: 0.8270 - val_loss: 0.7806 - val_acc: 0.8386 60 Epoch 29/1000 61 65s 131ms/step - loss: 0.8051 - acc: 0.8260 - val_loss: 0.7865 - val_acc: 0.8309 62 Epoch 30/1000 63 65s 131ms/step - loss: 0.8015 - acc: 0.8262 - val_loss: 0.7600 - val_acc: 0.8458 64 Epoch 31/1000 65 65s 131ms/step - loss: 0.7948 - acc: 0.8295 - val_loss: 0.7560 - val_acc: 0.8458 66 Epoch 32/1000 67 65s 131ms/step - loss: 0.7890 - acc: 0.8323 - val_loss: 0.7760 - val_acc: 0.8407 68 Epoch 33/1000 69 65s 131ms/step - loss: 0.7868 - acc: 0.8335 - val_loss: 0.7845 - val_acc: 0.8348 70 Epoch 34/1000 71 66s 131ms/step - loss: 0.7845 - acc: 0.8346 - val_loss: 0.7517 - val_acc: 0.8460 72 Epoch 35/1000 73 65s 131ms/step - loss: 0.7764 - acc: 0.8377 - val_loss: 0.7683 - val_acc: 0.8432 74 Epoch 36/1000 75 65s 131ms/step - loss: 0.7720 - acc: 0.8370 - val_loss: 0.7383 - val_acc: 0.8518 76 Epoch 37/1000 77 65s 131ms/step - loss: 0.7738 - acc: 0.8374 - val_loss: 0.7491 - val_acc: 0.8469 78 Epoch 38/1000 79 65s 131ms/step - loss: 0.7666 - acc: 0.8405 - val_loss: 0.7591 - val_acc: 0.8437 80 Epoch 39/1000 81 65s 131ms/step - loss: 0.7656 - acc: 0.8421 - val_loss: 0.7389 - val_acc: 0.8533 82 Epoch 40/1000 83 65s 131ms/step - loss: 0.7619 - acc: 0.8431 - val_loss: 0.7583 - val_acc: 0.8461 84 Epoch 41/1000 85 65s 130ms/step - loss: 0.7594 - acc: 0.8433 - val_loss: 0.7199 - val_acc: 0.8576 86 Epoch 42/1000 87 65s 131ms/step - loss: 0.7594 - acc: 0.8428 - val_loss: 0.7272 - val_acc: 0.8558 88 Epoch 43/1000 89 65s 131ms/step - loss: 0.7559 - acc: 0.8451 - val_loss: 0.7353 - val_acc: 0.8535 90 Epoch 44/1000 91 65s 131ms/step - loss: 0.7528 - acc: 0.8454 - val_loss: 0.7492 - val_acc: 0.8487 92 Epoch 45/1000 93 65s 131ms/step - loss: 0.7564 - acc: 0.8465 - val_loss: 0.7510 - val_acc: 0.8505 94 Epoch 46/1000 95 65s 131ms/step - loss: 0.7494 - acc: 0.8487 - val_loss: 0.7626 - val_acc: 0.8462 96 Epoch 47/1000 97 65s 131ms/step - loss: 0.7505 - acc: 0.8491 - val_loss: 0.7417 - val_acc: 0.8561 98 Epoch 48/1000 99 65s 131ms/step - loss: 0.7434 - acc: 0.8509 - val_loss: 0.7247 - val_acc: 0.8580 100 Epoch 49/1000 101 65s 131ms/step - loss: 0.7426 - acc: 0.8502 - val_loss: 0.7203 - val_acc: 0.8612 102 Epoch 50/1000 103 65s 130ms/step - loss: 0.7436 - acc: 0.8503 - val_loss: 0.7190 - val_acc: 0.8621 104 Epoch 51/1000 105 65s 130ms/step - loss: 0.7415 - acc: 0.8509 - val_loss: 0.7315 - val_acc: 0.8590 106 Epoch 52/1000 107 65s 130ms/step - loss: 0.7342 - acc: 0.8549 - val_loss: 0.7141 - val_acc: 0.8627 108 Epoch 53/1000 109 65s 130ms/step - loss: 0.7341 - acc: 0.8525 - val_loss: 0.7209 - val_acc: 0.8582 110 Epoch 54/1000 111 65s 130ms/step - loss: 0.7326 - acc: 0.8546 - val_loss: 0.7114 - val_acc: 0.8640 112 Epoch 55/1000 113 65s 131ms/step - loss: 0.7338 - acc: 0.8546 - val_loss: 0.7166 - val_acc: 0.8587 114 Epoch 56/1000 115 65s 131ms/step - loss: 0.7291 - acc: 0.8564 - val_loss: 0.7109 - val_acc: 0.8642 116 Epoch 57/1000 117 65s 131ms/step - loss: 0.7261 - acc: 0.8563 - val_loss: 0.7116 - val_acc: 0.8638 118 Epoch 58/1000 119 65s 131ms/step - loss: 0.7270 - acc: 0.8567 - val_loss: 0.7272 - val_acc: 0.8591 120 Epoch 59/1000 121 65s 131ms/step - loss: 0.7240 - acc: 0.8577 - val_loss: 0.6949 - val_acc: 0.8730 122 Epoch 60/1000 123 65s 130ms/step - loss: 0.7268 - acc: 0.8575 - val_loss: 0.7129 - val_acc: 0.8645 124 Epoch 61/1000 125 65s 131ms/step - loss: 0.7222 - acc: 0.8599 - val_loss: 0.7174 - val_acc: 0.8642 126 Epoch 62/1000 127 65s 131ms/step - loss: 0.7195 - acc: 0.8611 - val_loss: 0.7178 - val_acc: 0.8608 128 Epoch 63/1000 129 65s 131ms/step - loss: 0.7177 - acc: 0.8619 - val_loss: 0.7142 - val_acc: 0.8586 130 Epoch 64/1000 131 65s 131ms/step - loss: 0.7146 - acc: 0.8632 - val_loss: 0.7119 - val_acc: 0.8619 132 Epoch 65/1000 133 65s 131ms/step - loss: 0.7174 - acc: 0.8599 - val_loss: 0.7174 - val_acc: 0.8640 134 Epoch 66/1000 135 65s 131ms/step - loss: 0.7145 - acc: 0.8619 - val_loss: 0.7075 - val_acc: 0.8647 136 Epoch 67/1000 137 65s 131ms/step - loss: 0.7116 - acc: 0.8635 - val_loss: 0.7449 - val_acc: 0.8534 138 Epoch 68/1000 139 65s 131ms/step - loss: 0.7058 - acc: 0.8632 - val_loss: 0.6978 - val_acc: 0.8713 140 Epoch 69/1000 141 65s 131ms/step - loss: 0.7111 - acc: 0.8632 - val_loss: 0.7132 - val_acc: 0.8641 142 Epoch 70/1000 143 66s 131ms/step - loss: 0.7046 - acc: 0.8655 - val_loss: 0.6695 - val_acc: 0.8764 144 Epoch 71/1000 145 66s 131ms/step - loss: 0.7062 - acc: 0.8640 - val_loss: 0.6967 - val_acc: 0.8704 146 Epoch 72/1000 147 66s 131ms/step - loss: 0.7044 - acc: 0.8655 - val_loss: 0.6786 - val_acc: 0.8771 148 Epoch 73/1000 149 66s 131ms/step - loss: 0.7018 - acc: 0.8667 - val_loss: 0.7139 - val_acc: 0.8639 150 Epoch 74/1000 151 65s 131ms/step - loss: 0.7029 - acc: 0.8667 - val_loss: 0.7264 - val_acc: 0.8565 152 Epoch 75/1000 153 65s 131ms/step - loss: 0.6981 - acc: 0.8661 - val_loss: 0.6919 - val_acc: 0.8738 154 Epoch 76/1000 155 65s 131ms/step - loss: 0.6997 - acc: 0.8667 - val_loss: 0.7023 - val_acc: 0.8700 156 Epoch 77/1000 157 65s 131ms/step - loss: 0.6967 - acc: 0.8685 - val_loss: 0.6810 - val_acc: 0.8769 158 Epoch 78/1000 159 65s 131ms/step - loss: 0.6982 - acc: 0.8673 - val_loss: 0.7090 - val_acc: 0.8648 160 Epoch 79/1000 161 66s 131ms/step - loss: 0.6989 - acc: 0.8670 - val_loss: 0.7114 - val_acc: 0.8691 162 Epoch 80/1000 163 66s 131ms/step - loss: 0.6900 - acc: 0.8704 - val_loss: 0.7039 - val_acc: 0.8707 164 Epoch 81/1000 165 66s 131ms/step - loss: 0.6920 - acc: 0.8703 - val_loss: 0.6878 - val_acc: 0.8742 166 Epoch 82/1000 167 66s 131ms/step - loss: 0.6904 - acc: 0.8705 - val_loss: 0.6966 - val_acc: 0.8724 168 Epoch 83/1000 169 66s 131ms/step - loss: 0.6907 - acc: 0.8694 - val_loss: 0.6880 - val_acc: 0.8725 170 Epoch 84/1000 171 65s 131ms/step - loss: 0.6933 - acc: 0.8692 - val_loss: 0.7006 - val_acc: 0.8697 172 Epoch 85/1000 173 65s 131ms/step - loss: 0.6934 - acc: 0.8709 - val_loss: 0.7079 - val_acc: 0.8679 174 Epoch 86/1000 175 65s 131ms/step - loss: 0.6899 - acc: 0.8710 - val_loss: 0.7029 - val_acc: 0.8661 176 Epoch 87/1000 177 66s 131ms/step - loss: 0.6946 - acc: 0.8696 - val_loss: 0.6892 - val_acc: 0.8746 178 Epoch 88/1000 179 66s 131ms/step - loss: 0.6925 - acc: 0.8709 - val_loss: 0.6920 - val_acc: 0.8698 180 Epoch 89/1000 181 66s 131ms/step - loss: 0.6861 - acc: 0.8703 - val_loss: 0.6857 - val_acc: 0.8762 182 Epoch 90/1000 183 66s 131ms/step - loss: 0.6878 - acc: 0.8721 - val_loss: 0.6827 - val_acc: 0.8740 184 Epoch 91/1000 185 66s 131ms/step - loss: 0.6845 - acc: 0.8728 - val_loss: 0.6995 - val_acc: 0.8702 186 Epoch 92/1000 187 65s 131ms/step - loss: 0.6890 - acc: 0.8719 - val_loss: 0.6769 - val_acc: 0.8767 188 Epoch 93/1000 189 66s 131ms/step - loss: 0.6836 - acc: 0.8734 - val_loss: 0.6992 - val_acc: 0.8689 190 Epoch 94/1000 191 65s 131ms/step - loss: 0.6809 - acc: 0.8737 - val_loss: 0.7046 - val_acc: 0.8682 192 Epoch 95/1000 193 65s 131ms/step - loss: 0.6803 - acc: 0.8727 - val_loss: 0.6755 - val_acc: 0.8793 194 Epoch 96/1000 195 65s 131ms/step - loss: 0.6833 - acc: 0.8742 - val_loss: 0.6857 - val_acc: 0.8741 196 Epoch 97/1000 197 65s 131ms/step - loss: 0.6837 - acc: 0.8732 - val_loss: 0.6969 - val_acc: 0.8715 198 Epoch 98/1000 199 65s 131ms/step - loss: 0.6836 - acc: 0.8738 - val_loss: 0.6762 - val_acc: 0.8763 200 Epoch 99/1000 201 65s 131ms/step - loss: 0.6837 - acc: 0.8727 - val_loss: 0.6817 - val_acc: 0.8759 202 Epoch 100/1000 203 65s 131ms/step - loss: 0.6809 - acc: 0.8755 - val_loss: 0.6859 - val_acc: 0.8736 204 Epoch 101/1000 205 65s 131ms/step - loss: 0.6814 - acc: 0.8745 - val_loss: 0.6695 - val_acc: 0.8816 206 Epoch 102/1000 207 65s 131ms/step - loss: 0.6813 - acc: 0.8735 - val_loss: 0.6878 - val_acc: 0.8732 208 Epoch 103/1000 209 66s 131ms/step - loss: 0.6852 - acc: 0.8744 - val_loss: 0.6906 - val_acc: 0.8719 210 Epoch 104/1000 211 65s 131ms/step - loss: 0.6804 - acc: 0.8753 - val_loss: 0.6803 - val_acc: 0.8779 212 Epoch 105/1000 213 65s 131ms/step - loss: 0.6771 - acc: 0.8748 - val_loss: 0.6838 - val_acc: 0.8754 214 Epoch 106/1000 215 65s 131ms/step - loss: 0.6741 - acc: 0.8768 - val_loss: 0.7191 - val_acc: 0.8606 216 Epoch 107/1000 217 65s 131ms/step - loss: 0.6774 - acc: 0.8751 - val_loss: 0.6901 - val_acc: 0.8725 218 Epoch 108/1000 219 65s 131ms/step - loss: 0.6752 - acc: 0.8768 - val_loss: 0.7003 - val_acc: 0.8711 220 Epoch 109/1000 221 65s 130ms/step - loss: 0.6772 - acc: 0.8752 - val_loss: 0.6926 - val_acc: 0.8756 222 Epoch 110/1000 223 65s 131ms/step - loss: 0.6729 - acc: 0.8775 - val_loss: 0.7088 - val_acc: 0.8647 224 Epoch 111/1000 225 65s 131ms/step - loss: 0.6670 - acc: 0.8793 - val_loss: 0.6932 - val_acc: 0.8725 226 Epoch 112/1000 227 65s 131ms/step - loss: 0.6724 - acc: 0.8775 - val_loss: 0.6781 - val_acc: 0.8779 228 Epoch 113/1000 229 65s 131ms/step - loss: 0.6753 - acc: 0.8771 - val_loss: 0.6676 - val_acc: 0.8815 230 Epoch 114/1000 231 65s 131ms/step - loss: 0.6720 - acc: 0.8775 - val_loss: 0.6813 - val_acc: 0.8763 232 Epoch 115/1000 233 66s 131ms/step - loss: 0.6754 - acc: 0.8746 - val_loss: 0.6662 - val_acc: 0.8761 234 Epoch 116/1000 235 65s 130ms/step - loss: 0.6763 - acc: 0.8758 - val_loss: 0.6668 - val_acc: 0.8798 236 Epoch 117/1000 237 65s 131ms/step - loss: 0.6680 - acc: 0.8788 - val_loss: 0.6860 - val_acc: 0.8791 238 Epoch 118/1000 239 65s 131ms/step - loss: 0.6737 - acc: 0.8781 - val_loss: 0.6630 - val_acc: 0.8794 240 Epoch 119/1000 241 65s 131ms/step - loss: 0.6621 - acc: 0.8812 - val_loss: 0.6847 - val_acc: 0.8772 242 Epoch 120/1000 243 65s 131ms/step - loss: 0.6638 - acc: 0.8794 - val_loss: 0.6777 - val_acc: 0.8768 244 Epoch 121/1000 245 65s 131ms/step - loss: 0.6682 - acc: 0.8793 - val_loss: 0.7159 - val_acc: 0.8659 246 Epoch 122/1000 247 65s 131ms/step - loss: 0.6726 - acc: 0.8762 - val_loss: 0.6771 - val_acc: 0.8803 248 Epoch 123/1000 249 65s 131ms/step - loss: 0.6660 - acc: 0.8800 - val_loss: 0.6986 - val_acc: 0.8730 250 Epoch 124/1000 251 65s 131ms/step - loss: 0.6697 - acc: 0.8780 - val_loss: 0.6978 - val_acc: 0.8741 252 Epoch 125/1000 253 65s 131ms/step - loss: 0.6680 - acc: 0.8803 - val_loss: 0.6767 - val_acc: 0.8787 254 Epoch 126/1000 255 65s 131ms/step - loss: 0.6604 - acc: 0.8827 - val_loss: 0.6827 - val_acc: 0.8751 256 Epoch 127/1000 257 65s 131ms/step - loss: 0.6647 - acc: 0.8816 - val_loss: 0.7081 - val_acc: 0.8681 258 Epoch 128/1000 259 65s 130ms/step - loss: 0.6668 - acc: 0.8808 - val_loss: 0.6697 - val_acc: 0.8780 260 Epoch 129/1000 261 65s 131ms/step - loss: 0.6629 - acc: 0.8808 - val_loss: 0.6848 - val_acc: 0.8725 262 Epoch 130/1000 263 65s 131ms/step - loss: 0.6634 - acc: 0.8802 - val_loss: 0.6862 - val_acc: 0.8730 264 Epoch 131/1000 265 65s 131ms/step - loss: 0.6637 - acc: 0.8797 - val_loss: 0.7044 - val_acc: 0.8704 266 Epoch 132/1000 267 65s 131ms/step - loss: 0.6647 - acc: 0.8817 - val_loss: 0.6798 - val_acc: 0.8779 268 Epoch 133/1000 269 65s 131ms/step - loss: 0.6604 - acc: 0.8830 - val_loss: 0.6790 - val_acc: 0.8770 270 Epoch 134/1000 271 65s 131ms/step - loss: 0.6638 - acc: 0.8821 - val_loss: 0.6786 - val_acc: 0.8777 272 Epoch 135/1000 273 65s 131ms/step - loss: 0.6621 - acc: 0.8829 - val_loss: 0.6990 - val_acc: 0.8676 274 Epoch 136/1000 275 65s 131ms/step - loss: 0.6650 - acc: 0.8803 - val_loss: 0.6916 - val_acc: 0.8742 276 Epoch 137/1000 277 65s 131ms/step - loss: 0.6600 - acc: 0.8814 - val_loss: 0.6645 - val_acc: 0.8822 278 Epoch 138/1000 279 65s 131ms/step - loss: 0.6606 - acc: 0.8827 - val_loss: 0.6554 - val_acc: 0.8902 280 Epoch 139/1000 281 65s 131ms/step - loss: 0.6575 - acc: 0.8849 - val_loss: 0.6895 - val_acc: 0.8782 282 Epoch 140/1000 283 65s 131ms/step - loss: 0.6590 - acc: 0.8824 - val_loss: 0.6689 - val_acc: 0.8830 284 Epoch 141/1000 285 65s 131ms/step - loss: 0.6589 - acc: 0.8827 - val_loss: 0.6620 - val_acc: 0.8816 286 Epoch 142/1000 287 65s 131ms/step - loss: 0.6580 - acc: 0.8833 - val_loss: 0.6765 - val_acc: 0.8787 288 Epoch 143/1000 289 66s 131ms/step - loss: 0.6559 - acc: 0.8830 - val_loss: 0.7018 - val_acc: 0.8691 290 Epoch 144/1000 291 65s 131ms/step - loss: 0.6579 - acc: 0.8818 - val_loss: 0.6733 - val_acc: 0.8819 292 Epoch 145/1000 293 66s 131ms/step - loss: 0.6559 - acc: 0.8843 - val_loss: 0.6702 - val_acc: 0.8809 294 Epoch 146/1000 295 65s 131ms/step - loss: 0.6557 - acc: 0.8826 - val_loss: 0.6474 - val_acc: 0.8871 296 Epoch 147/1000 297 65s 131ms/step - loss: 0.6552 - acc: 0.8844 - val_loss: 0.6815 - val_acc: 0.8769 298 Epoch 148/1000 299 65s 131ms/step - loss: 0.6565 - acc: 0.8830 - val_loss: 0.6770 - val_acc: 0.8818 300 Epoch 149/1000 301 65s 131ms/step - loss: 0.6501 - acc: 0.8852 - val_loss: 0.6885 - val_acc: 0.8764 302 Epoch 150/1000 303 65s 131ms/step - loss: 0.6566 - acc: 0.8832 - val_loss: 0.6701 - val_acc: 0.8815 304 Epoch 151/1000 305 65s 131ms/step - loss: 0.6521 - acc: 0.8861 - val_loss: 0.6785 - val_acc: 0.8785 306 Epoch 152/1000 307 65s 131ms/step - loss: 0.6539 - acc: 0.8851 - val_loss: 0.6681 - val_acc: 0.8841 308 Epoch 153/1000 309 65s 131ms/step - loss: 0.6514 - acc: 0.8849 - val_loss: 0.6773 - val_acc: 0.8785 310 Epoch 154/1000 311 65s 131ms/step - loss: 0.6561 - acc: 0.8836 - val_loss: 0.6747 - val_acc: 0.8803 312 Epoch 155/1000 313 65s 131ms/step - loss: 0.6524 - acc: 0.8852 - val_loss: 0.6545 - val_acc: 0.8854 314 Epoch 156/1000 315 65s 131ms/step - loss: 0.6587 - acc: 0.8828 - val_loss: 0.7070 - val_acc: 0.8692 316 Epoch 157/1000 317 65s 131ms/step - loss: 0.6558 - acc: 0.8838 - val_loss: 0.6618 - val_acc: 0.8843 318 Epoch 158/1000 319 65s 131ms/step - loss: 0.6514 - acc: 0.8873 - val_loss: 0.6874 - val_acc: 0.8763 320 Epoch 159/1000 321 65s 131ms/step - loss: 0.6564 - acc: 0.8848 - val_loss: 0.6804 - val_acc: 0.8805 322 Epoch 160/1000 323 65s 131ms/step - loss: 0.6450 - acc: 0.8868 - val_loss: 0.6752 - val_acc: 0.8800 324 Epoch 161/1000 325 65s 131ms/step - loss: 0.6555 - acc: 0.8847 - val_loss: 0.6589 - val_acc: 0.8857 326 Epoch 162/1000 327 65s 131ms/step - loss: 0.6492 - acc: 0.8860 - val_loss: 0.6544 - val_acc: 0.8862 328 Epoch 163/1000 329 65s 131ms/step - loss: 0.6544 - acc: 0.8844 - val_loss: 0.6807 - val_acc: 0.8775 330 Epoch 164/1000 331 65s 131ms/step - loss: 0.6504 - acc: 0.8850 - val_loss: 0.6861 - val_acc: 0.8761 332 Epoch 165/1000 333 65s 131ms/step - loss: 0.6538 - acc: 0.8832 - val_loss: 0.6653 - val_acc: 0.8842 334 Epoch 166/1000 335 65s 131ms/step - loss: 0.6520 - acc: 0.8866 - val_loss: 0.6685 - val_acc: 0.8823 336 Epoch 167/1000 337 65s 131ms/step - loss: 0.6483 - acc: 0.8869 - val_loss: 0.6916 - val_acc: 0.8719 338 Epoch 168/1000 339 65s 131ms/step - loss: 0.6501 - acc: 0.8855 - val_loss: 0.6789 - val_acc: 0.8785 340 Epoch 169/1000 341 65s 131ms/step - loss: 0.6484 - acc: 0.8863 - val_loss: 0.6853 - val_acc: 0.8740 342 Epoch 170/1000 343 65s 131ms/step - loss: 0.6485 - acc: 0.8863 - val_loss: 0.6654 - val_acc: 0.8808 344 Epoch 171/1000 345 65s 131ms/step - loss: 0.6474 - acc: 0.8863 - val_loss: 0.6636 - val_acc: 0.8858 346 Epoch 172/1000 347 65s 131ms/step - loss: 0.6469 - acc: 0.8863 - val_loss: 0.6752 - val_acc: 0.8793 348 Epoch 173/1000 349 65s 131ms/step - loss: 0.6411 - acc: 0.8886 - val_loss: 0.6869 - val_acc: 0.8769 350 Epoch 174/1000 351 65s 130ms/step - loss: 0.6456 - acc: 0.8873 - val_loss: 0.6714 - val_acc: 0.8808 352 Epoch 175/1000 353 65s 130ms/step - loss: 0.6536 - acc: 0.8853 - val_loss: 0.6580 - val_acc: 0.8885 354 Epoch 176/1000 355 65s 130ms/step - loss: 0.6491 - acc: 0.8857 - val_loss: 0.6743 - val_acc: 0.8816 356 Epoch 177/1000 357 65s 130ms/step - loss: 0.6492 - acc: 0.8851 - val_loss: 0.6625 - val_acc: 0.8897 358 Epoch 178/1000 359 65s 130ms/step - loss: 0.6481 - acc: 0.8845 - val_loss: 0.6671 - val_acc: 0.8826 360 Epoch 179/1000 361 65s 131ms/step - loss: 0.6495 - acc: 0.8854 - val_loss: 0.6968 - val_acc: 0.8724 362 Epoch 180/1000 363 65s 131ms/step - loss: 0.6474 - acc: 0.8879 - val_loss: 0.6602 - val_acc: 0.8860 364 Epoch 181/1000 365 65s 131ms/step - loss: 0.6449 - acc: 0.8869 - val_loss: 0.6648 - val_acc: 0.8849 366 Epoch 182/1000 367 65s 131ms/step - loss: 0.6515 - acc: 0.8849 - val_loss: 0.6675 - val_acc: 0.8812 368 Epoch 183/1000 369 65s 131ms/step - loss: 0.6489 - acc: 0.8861 - val_loss: 0.6561 - val_acc: 0.8863 370 Epoch 184/1000 371 65s 131ms/step - loss: 0.6435 - acc: 0.8892 - val_loss: 0.6526 - val_acc: 0.8894 372 Epoch 185/1000 373 65s 131ms/step - loss: 0.6471 - acc: 0.8868 - val_loss: 0.6856 - val_acc: 0.8758 374 Epoch 186/1000 375 65s 131ms/step - loss: 0.6525 - acc: 0.8854 - val_loss: 0.6785 - val_acc: 0.8781 376 Epoch 187/1000 377 65s 131ms/step - loss: 0.6489 - acc: 0.8850 - val_loss: 0.6638 - val_acc: 0.8832 378 Epoch 188/1000 379 65s 131ms/step - loss: 0.6454 - acc: 0.8872 - val_loss: 0.6673 - val_acc: 0.8841 380 Epoch 189/1000 381 65s 131ms/step - loss: 0.6491 - acc: 0.8868 - val_loss: 0.6410 - val_acc: 0.8893 382 Epoch 190/1000 383 65s 131ms/step - loss: 0.6428 - acc: 0.8884 - val_loss: 0.6678 - val_acc: 0.8835 384 Epoch 191/1000 385 65s 131ms/step - loss: 0.6463 - acc: 0.8871 - val_loss: 0.6676 - val_acc: 0.8854 386 Epoch 192/1000 387 65s 131ms/step - loss: 0.6435 - acc: 0.8892 - val_loss: 0.6869 - val_acc: 0.8764 388 Epoch 193/1000 389 65s 131ms/step - loss: 0.6465 - acc: 0.8877 - val_loss: 0.6578 - val_acc: 0.8849 390 Epoch 194/1000 391 65s 131ms/step - loss: 0.6446 - acc: 0.8879 - val_loss: 0.6819 - val_acc: 0.8825 392 Epoch 195/1000 393 65s 131ms/step - loss: 0.6464 - acc: 0.8868 - val_loss: 0.6682 - val_acc: 0.8831 394 Epoch 196/1000 395 65s 131ms/step - loss: 0.6455 - acc: 0.8888 - val_loss: 0.6580 - val_acc: 0.8863 396 Epoch 197/1000 397 65s 131ms/step - loss: 0.6408 - acc: 0.8883 - val_loss: 0.6818 - val_acc: 0.8778 398 Epoch 198/1000 399 65s 131ms/step - loss: 0.6415 - acc: 0.8887 - val_loss: 0.6616 - val_acc: 0.8856 400 Epoch 199/1000 401 65s 131ms/step - loss: 0.6429 - acc: 0.8897 - val_loss: 0.6876 - val_acc: 0.8769 402 Epoch 200/1000 403 66s 131ms/step - loss: 0.6490 - acc: 0.8857 - val_loss: 0.6679 - val_acc: 0.8827 404 Epoch 201/1000 405 65s 131ms/step - loss: 0.6403 - acc: 0.8905 - val_loss: 0.6663 - val_acc: 0.8818 406 Epoch 202/1000 407 66s 131ms/step - loss: 0.6407 - acc: 0.8900 - val_loss: 0.6714 - val_acc: 0.8789 408 Epoch 203/1000 409 66s 131ms/step - loss: 0.6380 - acc: 0.8906 - val_loss: 0.6718 - val_acc: 0.8799 410 Epoch 204/1000 411 65s 131ms/step - loss: 0.6422 - acc: 0.8882 - val_loss: 0.6778 - val_acc: 0.8770 412 Epoch 205/1000 413 65s 129ms/step - loss: 0.6392 - acc: 0.8894 - val_loss: 0.6697 - val_acc: 0.8805 414 Epoch 206/1000 415 65s 129ms/step - loss: 0.6467 - acc: 0.8882 - val_loss: 0.6956 - val_acc: 0.8737 416 Epoch 207/1000 417 65s 131ms/step - loss: 0.6391 - acc: 0.8902 - val_loss: 0.6641 - val_acc: 0.8849 418 Epoch 208/1000 419 65s 131ms/step - loss: 0.6378 - acc: 0.8900 - val_loss: 0.6890 - val_acc: 0.8733 420 Epoch 209/1000 421 65s 131ms/step - loss: 0.6421 - acc: 0.8897 - val_loss: 0.6654 - val_acc: 0.8824 422 Epoch 210/1000 423 65s 131ms/step - loss: 0.6405 - acc: 0.8892 - val_loss: 0.6685 - val_acc: 0.8793 424 Epoch 211/1000 425 65s 131ms/step - loss: 0.6381 - acc: 0.8893 - val_loss: 0.6581 - val_acc: 0.8855 426 Epoch 212/1000 427 65s 131ms/step - loss: 0.6379 - acc: 0.8915 - val_loss: 0.6626 - val_acc: 0.8893 428 Epoch 213/1000 429 65s 131ms/step - loss: 0.6405 - acc: 0.8892 - val_loss: 0.6688 - val_acc: 0.8803 430 Epoch 214/1000 431 65s 131ms/step - loss: 0.6369 - acc: 0.8896 - val_loss: 0.6827 - val_acc: 0.8770 432 Epoch 215/1000 433 65s 131ms/step - loss: 0.6412 - acc: 0.8892 - val_loss: 0.6545 - val_acc: 0.8849 434 Epoch 216/1000 435 65s 131ms/step - loss: 0.6383 - acc: 0.8901 - val_loss: 0.6683 - val_acc: 0.8836 436 Epoch 217/1000 437 65s 131ms/step - loss: 0.6369 - acc: 0.8901 - val_loss: 0.6657 - val_acc: 0.8854 438 Epoch 218/1000 439 65s 131ms/step - loss: 0.6408 - acc: 0.8896 - val_loss: 0.6496 - val_acc: 0.8864 440 Epoch 219/1000 441 65s 131ms/step - loss: 0.6391 - acc: 0.8900 - val_loss: 0.6728 - val_acc: 0.8818 442 Epoch 220/1000 443 65s 131ms/step - loss: 0.6352 - acc: 0.8905 - val_loss: 0.6821 - val_acc: 0.8817 444 Epoch 221/1000 445 65s 131ms/step - loss: 0.6365 - acc: 0.8919 - val_loss: 0.6650 - val_acc: 0.8845 446 Epoch 222/1000 447 65s 131ms/step - loss: 0.6389 - acc: 0.8907 - val_loss: 0.6509 - val_acc: 0.8870 448 Epoch 223/1000 449 65s 131ms/step - loss: 0.6364 - acc: 0.8911 - val_loss: 0.6672 - val_acc: 0.8853 450 Epoch 224/1000 451 65s 131ms/step - loss: 0.6329 - acc: 0.8909 - val_loss: 0.6668 - val_acc: 0.8819 452 Epoch 225/1000 453 65s 131ms/step - loss: 0.6349 - acc: 0.8918 - val_loss: 0.6517 - val_acc: 0.8890 454 Epoch 226/1000 455 65s 131ms/step - loss: 0.6383 - acc: 0.8901 - val_loss: 0.6778 - val_acc: 0.8791 456 Epoch 227/1000 457 65s 131ms/step - loss: 0.6375 - acc: 0.8907 - val_loss: 0.6692 - val_acc: 0.8836 458 Epoch 228/1000 459 65s 131ms/step - loss: 0.6354 - acc: 0.8914 - val_loss: 0.6800 - val_acc: 0.8805 460 Epoch 229/1000 461 65s 131ms/step - loss: 0.6373 - acc: 0.8915 - val_loss: 0.6575 - val_acc: 0.8852 462 Epoch 230/1000 463 65s 131ms/step - loss: 0.6388 - acc: 0.8894 - val_loss: 0.6676 - val_acc: 0.8846 464 Epoch 231/1000 465 65s 131ms/step - loss: 0.6374 - acc: 0.8916 - val_loss: 0.6638 - val_acc: 0.8841 466 Epoch 232/1000 467 66s 132ms/step - loss: 0.6367 - acc: 0.8925 - val_loss: 0.6715 - val_acc: 0.8851 468 Epoch 233/1000 469 65s 131ms/step - loss: 0.6407 - acc: 0.8894 - val_loss: 0.6633 - val_acc: 0.8862 470 Epoch 234/1000 471 66s 131ms/step - loss: 0.6320 - acc: 0.8936 - val_loss: 0.6821 - val_acc: 0.8789 472 Epoch 235/1000 473 65s 131ms/step - loss: 0.6376 - acc: 0.8914 - val_loss: 0.6735 - val_acc: 0.8812 474 Epoch 236/1000 475 65s 131ms/step - loss: 0.6353 - acc: 0.8904 - val_loss: 0.6680 - val_acc: 0.8871 476 Epoch 237/1000 477 65s 131ms/step - loss: 0.6357 - acc: 0.8913 - val_loss: 0.6624 - val_acc: 0.8864 478 Epoch 238/1000 479 65s 131ms/step - loss: 0.6310 - acc: 0.8936 - val_loss: 0.6616 - val_acc: 0.8832 480 Epoch 239/1000 481 65s 131ms/step - loss: 0.6383 - acc: 0.8902 - val_loss: 0.6663 - val_acc: 0.8842 482 Epoch 240/1000 483 65s 131ms/step - loss: 0.6337 - acc: 0.8932 - val_loss: 0.6471 - val_acc: 0.8892 484 Epoch 241/1000 485 65s 131ms/step - loss: 0.6311 - acc: 0.8921 - val_loss: 0.6608 - val_acc: 0.8853 486 Epoch 242/1000 487 65s 131ms/step - loss: 0.6373 - acc: 0.8899 - val_loss: 0.6988 - val_acc: 0.8710 488 Epoch 243/1000 489 65s 131ms/step - loss: 0.6369 - acc: 0.8905 - val_loss: 0.6644 - val_acc: 0.8843 490 Epoch 244/1000 491 65s 130ms/step - loss: 0.6317 - acc: 0.8927 - val_loss: 0.6922 - val_acc: 0.8721 492 Epoch 245/1000 493 65s 131ms/step - loss: 0.6304 - acc: 0.8929 - val_loss: 0.6733 - val_acc: 0.8798 494 Epoch 246/1000 495 65s 131ms/step - loss: 0.6328 - acc: 0.8912 - val_loss: 0.6564 - val_acc: 0.8860 496 Epoch 247/1000 497 65s 131ms/step - loss: 0.6400 - acc: 0.8896 - val_loss: 0.6664 - val_acc: 0.8794 498 Epoch 248/1000 499 65s 131ms/step - loss: 0.6361 - acc: 0.8898 - val_loss: 0.6896 - val_acc: 0.8777 500 Epoch 249/1000 501 65s 131ms/step - loss: 0.6332 - acc: 0.8914 - val_loss: 0.6707 - val_acc: 0.8829 502 Epoch 250/1000 503 65s 131ms/step - loss: 0.6348 - acc: 0.8901 - val_loss: 0.6581 - val_acc: 0.8850 504 Epoch 251/1000 505 65s 131ms/step - loss: 0.6325 - acc: 0.8918 - val_loss: 0.6623 - val_acc: 0.8870 506 Epoch 252/1000 507 65s 131ms/step - loss: 0.6337 - acc: 0.8915 - val_loss: 0.6795 - val_acc: 0.8806 508 Epoch 253/1000 509 65s 131ms/step - loss: 0.6339 - acc: 0.8909 - val_loss: 0.6760 - val_acc: 0.8788 510 Epoch 254/1000 511 65s 131ms/step - loss: 0.6350 - acc: 0.8907 - val_loss: 0.6667 - val_acc: 0.8863 512 Epoch 255/1000 513 65s 131ms/step - loss: 0.6312 - acc: 0.8927 - val_loss: 0.6825 - val_acc: 0.8775 514 Epoch 256/1000 515 65s 131ms/step - loss: 0.6304 - acc: 0.8920 - val_loss: 0.6648 - val_acc: 0.8839 516 Epoch 257/1000 517 65s 131ms/step - loss: 0.6317 - acc: 0.8917 - val_loss: 0.6624 - val_acc: 0.8830 518 Epoch 258/1000 519 65s 131ms/step - loss: 0.6335 - acc: 0.8914 - val_loss: 0.6547 - val_acc: 0.8877 520 Epoch 259/1000 521 65s 131ms/step - loss: 0.6346 - acc: 0.8903 - val_loss: 0.6671 - val_acc: 0.8863 522 Epoch 260/1000 523 65s 131ms/step - loss: 0.6303 - acc: 0.8909 - val_loss: 0.6491 - val_acc: 0.8862 524 Epoch 261/1000 525 65s 131ms/step - loss: 0.6348 - acc: 0.8902 - val_loss: 0.6778 - val_acc: 0.8781 526 Epoch 262/1000 527 65s 131ms/step - loss: 0.6325 - acc: 0.8928 - val_loss: 0.6651 - val_acc: 0.8800 528 Epoch 263/1000 529 65s 131ms/step - loss: 0.6377 - acc: 0.8895 - val_loss: 0.6474 - val_acc: 0.8908 530 Epoch 264/1000 531 65s 131ms/step - loss: 0.6293 - acc: 0.8927 - val_loss: 0.6707 - val_acc: 0.8821 532 Epoch 265/1000 533 65s 131ms/step - loss: 0.6321 - acc: 0.8915 - val_loss: 0.6679 - val_acc: 0.8820 534 Epoch 266/1000 535 65s 131ms/step - loss: 0.6323 - acc: 0.8936 - val_loss: 0.6647 - val_acc: 0.8851 536 Epoch 267/1000 537 65s 131ms/step - loss: 0.6311 - acc: 0.8926 - val_loss: 0.6748 - val_acc: 0.8786 538 Epoch 268/1000 539 65s 131ms/step - loss: 0.6344 - acc: 0.8920 - val_loss: 0.6851 - val_acc: 0.8826 540 Epoch 269/1000 541 65s 131ms/step - loss: 0.6296 - acc: 0.8943 - val_loss: 0.6626 - val_acc: 0.8854 542 Epoch 270/1000 543 65s 131ms/step - loss: 0.6323 - acc: 0.8931 - val_loss: 0.6555 - val_acc: 0.8864 544 Epoch 271/1000 545 65s 131ms/step - loss: 0.6285 - acc: 0.8933 - val_loss: 0.6781 - val_acc: 0.8817 546 Epoch 272/1000 547 65s 131ms/step - loss: 0.6316 - acc: 0.8921 - val_loss: 0.6630 - val_acc: 0.8870 548 Epoch 273/1000 549 65s 131ms/step - loss: 0.6296 - acc: 0.8943 - val_loss: 0.6682 - val_acc: 0.8824 550 Epoch 274/1000 551 65s 131ms/step - loss: 0.6221 - acc: 0.8957 - val_loss: 0.6788 - val_acc: 0.8791 552 Epoch 275/1000 553 65s 131ms/step - loss: 0.6317 - acc: 0.8918 - val_loss: 0.6434 - val_acc: 0.8917 554 Epoch 276/1000 555 65s 130ms/step - loss: 0.6290 - acc: 0.8927 - val_loss: 0.6572 - val_acc: 0.8868 556 Epoch 277/1000 557 65s 131ms/step - loss: 0.6279 - acc: 0.8931 - val_loss: 0.6877 - val_acc: 0.8757 558 Epoch 278/1000 559 65s 131ms/step - loss: 0.6301 - acc: 0.8923 - val_loss: 0.6746 - val_acc: 0.8770 560 Epoch 279/1000 561 65s 131ms/step - loss: 0.6334 - acc: 0.8919 - val_loss: 0.6553 - val_acc: 0.8863 562 Epoch 280/1000 563 65s 131ms/step - loss: 0.6320 - acc: 0.8927 - val_loss: 0.6727 - val_acc: 0.8812 564 Epoch 281/1000 565 65s 131ms/step - loss: 0.6290 - acc: 0.8944 - val_loss: 0.6784 - val_acc: 0.8765 566 Epoch 282/1000 567 65s 131ms/step - loss: 0.6290 - acc: 0.8937 - val_loss: 0.6466 - val_acc: 0.8924 568 Epoch 283/1000 569 65s 131ms/step - loss: 0.6297 - acc: 0.8940 - val_loss: 0.6622 - val_acc: 0.8853 570 Epoch 284/1000 571 65s 131ms/step - loss: 0.6267 - acc: 0.8940 - val_loss: 0.6592 - val_acc: 0.8860 572 Epoch 285/1000 573 65s 131ms/step - loss: 0.6319 - acc: 0.8926 - val_loss: 0.6628 - val_acc: 0.8849 574 Epoch 286/1000 575 65s 131ms/step - loss: 0.6314 - acc: 0.8935 - val_loss: 0.6617 - val_acc: 0.8855 576 Epoch 287/1000 577 65s 131ms/step - loss: 0.6251 - acc: 0.8949 - val_loss: 0.6846 - val_acc: 0.8761 578 Epoch 288/1000 579 65s 131ms/step - loss: 0.6311 - acc: 0.8923 - val_loss: 0.6675 - val_acc: 0.8826 580 Epoch 289/1000 581 65s 131ms/step - loss: 0.6282 - acc: 0.8938 - val_loss: 0.6756 - val_acc: 0.8799 582 Epoch 290/1000 583 65s 131ms/step - loss: 0.6289 - acc: 0.8938 - val_loss: 0.6717 - val_acc: 0.8831 584 Epoch 291/1000 585 65s 131ms/step - loss: 0.6288 - acc: 0.8926 - val_loss: 0.6444 - val_acc: 0.8908 586 Epoch 292/1000 587 65s 131ms/step - loss: 0.6257 - acc: 0.8943 - val_loss: 0.6434 - val_acc: 0.8882 588 Epoch 293/1000 589 65s 131ms/step - loss: 0.6269 - acc: 0.8926 - val_loss: 0.6450 - val_acc: 0.8896 590 Epoch 294/1000 591 65s 131ms/step - loss: 0.6284 - acc: 0.8929 - val_loss: 0.6520 - val_acc: 0.8855 592 Epoch 295/1000 593 65s 131ms/step - loss: 0.6234 - acc: 0.8941 - val_loss: 0.6519 - val_acc: 0.8899 594 Epoch 296/1000 595 66s 131ms/step - loss: 0.6284 - acc: 0.8935 - val_loss: 0.6571 - val_acc: 0.8827 596 Epoch 297/1000 597 65s 131ms/step - loss: 0.6265 - acc: 0.8940 - val_loss: 0.6566 - val_acc: 0.8857 598 Epoch 298/1000 599 65s 131ms/step - loss: 0.6287 - acc: 0.8936 - val_loss: 0.6573 - val_acc: 0.8841 600 Epoch 299/1000 601 65s 131ms/step - loss: 0.6237 - acc: 0.8954 - val_loss: 0.6371 - val_acc: 0.8937 602 Epoch 300/1000 603 65s 131ms/step - loss: 0.6263 - acc: 0.8943 - val_loss: 0.6537 - val_acc: 0.8884 604 Epoch 301/1000 605 lr changed to 0.010000000149011612 606 65s 131ms/step - loss: 0.5256 - acc: 0.9298 - val_loss: 0.5518 - val_acc: 0.9215 607 Epoch 302/1000 608 66s 131ms/step - loss: 0.4681 - acc: 0.9470 - val_loss: 0.5407 - val_acc: 0.9233 609 Epoch 303/1000 610 66s 131ms/step - loss: 0.4455 - acc: 0.9532 - val_loss: 0.5319 - val_acc: 0.9258 611 Epoch 304/1000 612 65s 131ms/step - loss: 0.4308 - acc: 0.9559 - val_loss: 0.5251 - val_acc: 0.9277 613 Epoch 305/1000 614 65s 131ms/step - loss: 0.4180 - acc: 0.9595 - val_loss: 0.5182 - val_acc: 0.9290 615 Epoch 306/1000 616 65s 131ms/step - loss: 0.4088 - acc: 0.9609 - val_loss: 0.5124 - val_acc: 0.9300 617 Epoch 307/1000 618 65s 131ms/step - loss: 0.3970 - acc: 0.9628 - val_loss: 0.5158 - val_acc: 0.9277 619 Epoch 308/1000 620 65s 131ms/step - loss: 0.3877 - acc: 0.9653 - val_loss: 0.5093 - val_acc: 0.9298 621 Epoch 309/1000 622 65s 131ms/step - loss: 0.3794 - acc: 0.9664 - val_loss: 0.5062 - val_acc: 0.9281 623 Epoch 310/1000 624 65s 131ms/step - loss: 0.3736 - acc: 0.9666 - val_loss: 0.5056 - val_acc: 0.9267 625 Epoch 311/1000 626 65s 131ms/step - loss: 0.3675 - acc: 0.9669 - val_loss: 0.4959 - val_acc: 0.9295 627 Epoch 312/1000 628 65s 131ms/step - loss: 0.3631 - acc: 0.9670 - val_loss: 0.4913 - val_acc: 0.9313 629 Epoch 313/1000 630 65s 131ms/step - loss: 0.3538 - acc: 0.9686 - val_loss: 0.4924 - val_acc: 0.9299 631 Epoch 314/1000 632 65s 131ms/step - loss: 0.3432 - acc: 0.9716 - val_loss: 0.4920 - val_acc: 0.9296 633 Epoch 315/1000 634 65s 131ms/step - loss: 0.3434 - acc: 0.9701 - val_loss: 0.4838 - val_acc: 0.9277 635 Epoch 316/1000 636 65s 131ms/step - loss: 0.3325 - acc: 0.9719 - val_loss: 0.4822 - val_acc: 0.9301 637 Epoch 317/1000 638 65s 130ms/step - loss: 0.3283 - acc: 0.9724 - val_loss: 0.4882 - val_acc: 0.9270 639 Epoch 318/1000 640 65s 129ms/step - loss: 0.3259 - acc: 0.9727 - val_loss: 0.4866 - val_acc: 0.9263 641 Epoch 319/1000 642 65s 130ms/step - loss: 0.3200 - acc: 0.9728 - val_loss: 0.4780 - val_acc: 0.9279 643 Epoch 320/1000 644 65s 131ms/step - loss: 0.3156 - acc: 0.9733 - val_loss: 0.4768 - val_acc: 0.9256 645 Epoch 321/1000 646 65s 131ms/step - loss: 0.3109 - acc: 0.9738 - val_loss: 0.4662 - val_acc: 0.9274 647 Epoch 322/1000 648 65s 131ms/step - loss: 0.3070 - acc: 0.9743 - val_loss: 0.4666 - val_acc: 0.9266 649 Epoch 323/1000 650 65s 131ms/step - loss: 0.3008 - acc: 0.9754 - val_loss: 0.4734 - val_acc: 0.9244 651 Epoch 324/1000 652 65s 131ms/step - loss: 0.3005 - acc: 0.9739 - val_loss: 0.4770 - val_acc: 0.9276 653 Epoch 325/1000 654 65s 131ms/step - loss: 0.2967 - acc: 0.9736 - val_loss: 0.4575 - val_acc: 0.9289 655 Epoch 326/1000 656 65s 131ms/step - loss: 0.2945 - acc: 0.9742 - val_loss: 0.4677 - val_acc: 0.9247 657 Epoch 327/1000 658 65s 131ms/step - loss: 0.2862 - acc: 0.9760 - val_loss: 0.4682 - val_acc: 0.9263 659 Epoch 328/1000 660 65s 131ms/step - loss: 0.2850 - acc: 0.9762 - val_loss: 0.4657 - val_acc: 0.9247 661 Epoch 329/1000 662 65s 131ms/step - loss: 0.2816 - acc: 0.9757 - val_loss: 0.4617 - val_acc: 0.9265 663 Epoch 330/1000 664 65s 131ms/step - loss: 0.2812 - acc: 0.9744 - val_loss: 0.4649 - val_acc: 0.9226 665 Epoch 331/1000 666 65s 131ms/step - loss: 0.2791 - acc: 0.9744 - val_loss: 0.4484 - val_acc: 0.9282 667 Epoch 332/1000 668 65s 131ms/step - loss: 0.2743 - acc: 0.9757 - val_loss: 0.4503 - val_acc: 0.9242 669 Epoch 333/1000 670 65s 131ms/step - loss: 0.2706 - acc: 0.9767 - val_loss: 0.4464 - val_acc: 0.9295 671 Epoch 334/1000 672 65s 131ms/step - loss: 0.2690 - acc: 0.9757 - val_loss: 0.4507 - val_acc: 0.9272 673 Epoch 335/1000 674 65s 131ms/step - loss: 0.2649 - acc: 0.9762 - val_loss: 0.4510 - val_acc: 0.9246 675 Epoch 336/1000 676 66s 131ms/step - loss: 0.2626 - acc: 0.9776 - val_loss: 0.4529 - val_acc: 0.9226 677 Epoch 337/1000 678 66s 131ms/step - loss: 0.2615 - acc: 0.9772 - val_loss: 0.4453 - val_acc: 0.9270 679 Epoch 338/1000 680 66s 131ms/step - loss: 0.2597 - acc: 0.9763 - val_loss: 0.4571 - val_acc: 0.9232 681 Epoch 339/1000 682 65s 131ms/step - loss: 0.2555 - acc: 0.9776 - val_loss: 0.4449 - val_acc: 0.9247 683 ... 684 Epoch 755/1000 685 65s 130ms/step - loss: 0.1093 - acc: 0.9992 - val_loss: 0.3584 - val_acc: 0.9337 686 Epoch 756/1000 687 65s 130ms/step - loss: 0.1093 - acc: 0.9990 - val_loss: 0.3583 - val_acc: 0.9346 688 Epoch 757/1000 689 65s 130ms/step - loss: 0.1095 - acc: 0.9991 - val_loss: 0.3560 - val_acc: 0.9346 690 Epoch 758/1000 691 65s 130ms/step - loss: 0.1090 - acc: 0.9991 - val_loss: 0.3587 - val_acc: 0.9346 692 Epoch 759/1000 693 65s 130ms/step - loss: 0.1092 - acc: 0.9989 - val_loss: 0.3594 - val_acc: 0.9346 694 Epoch 760/1000 695 65s 130ms/step - loss: 0.1086 - acc: 0.9992 - val_loss: 0.3560 - val_acc: 0.9345 696 Epoch 761/1000 697 65s 130ms/step - loss: 0.1081 - acc: 0.9993 - val_loss: 0.3573 - val_acc: 0.9346 698 Epoch 762/1000 699 65s 129ms/step - loss: 0.1083 - acc: 0.9992 - val_loss: 0.3598 - val_acc: 0.9343 700 Epoch 763/1000 701 65s 130ms/step - loss: 0.1080 - acc: 0.9991 - val_loss: 0.3590 - val_acc: 0.9341 702 Epoch 764/1000 703 65s 130ms/step - loss: 0.1076 - acc: 0.9993 - val_loss: 0.3567 - val_acc: 0.9336 704 Epoch 765/1000 705 65s 130ms/step - loss: 0.1077 - acc: 0.9991 - val_loss: 0.3556 - val_acc: 0.9375 706 Epoch 766/1000 707 65s 130ms/step - loss: 0.1072 - acc: 0.9993 - val_loss: 0.3562 - val_acc: 0.9349 708 Epoch 767/1000 709 65s 130ms/step - loss: 0.1075 - acc: 0.9992 - val_loss: 0.3538 - val_acc: 0.9364 710 Epoch 768/1000 711 65s 130ms/step - loss: 0.1071 - acc: 0.9991 - val_loss: 0.3607 - val_acc: 0.9347 712 Epoch 769/1000 713 65s 130ms/step - loss: 0.1067 - acc: 0.9994 - val_loss: 0.3626 - val_acc: 0.9348 714 Epoch 770/1000 715 65s 130ms/step - loss: 0.1070 - acc: 0.9991 - val_loss: 0.3595 - val_acc: 0.9364 716 Epoch 771/1000 717 65s 130ms/step - loss: 0.1067 - acc: 0.9991 - val_loss: 0.3578 - val_acc: 0.9353 718 Epoch 772/1000 719 65s 130ms/step - loss: 0.1066 - acc: 0.9991 - val_loss: 0.3561 - val_acc: 0.9357 720 Epoch 773/1000 721 65s 130ms/step - loss: 0.1062 - acc: 0.9992 - val_loss: 0.3555 - val_acc: 0.9357 722 Epoch 774/1000 723 65s 130ms/step - loss: 0.1062 - acc: 0.9992 - val_loss: 0.3546 - val_acc: 0.9367 724 Epoch 775/1000 725 65s 130ms/step - loss: 0.1059 - acc: 0.9992 - val_loss: 0.3570 - val_acc: 0.9367 726 Epoch 776/1000 727 65s 130ms/step - loss: 0.1061 - acc: 0.9990 - val_loss: 0.3570 - val_acc: 0.9355 728 Epoch 777/1000 729 65s 129ms/step - loss: 0.1065 - acc: 0.9988 - val_loss: 0.3569 - val_acc: 0.9361 730 Epoch 778/1000 731 65s 129ms/step - loss: 0.1055 - acc: 0.9991 - val_loss: 0.3592 - val_acc: 0.9347 732 Epoch 779/1000 733 65s 129ms/step - loss: 0.1053 - acc: 0.9991 - val_loss: 0.3578 - val_acc: 0.9345 734 Epoch 780/1000 735 65s 130ms/step - loss: 0.1057 - acc: 0.9990 - val_loss: 0.3550 - val_acc: 0.9361 736 Epoch 781/1000 737 65s 130ms/step - loss: 0.1054 - acc: 0.9988 - val_loss: 0.3598 - val_acc: 0.9359 738 Epoch 782/1000 739 65s 130ms/step - loss: 0.1053 - acc: 0.9988 - val_loss: 0.3548 - val_acc: 0.9349 740 Epoch 783/1000 741 65s 129ms/step - loss: 0.1047 - acc: 0.9992 - val_loss: 0.3541 - val_acc: 0.9366 742 Epoch 784/1000 743 65s 130ms/step - loss: 0.1048 - acc: 0.9990 - val_loss: 0.3540 - val_acc: 0.9346 744 Epoch 785/1000 745 65s 130ms/step - loss: 0.1046 - acc: 0.9991 - val_loss: 0.3534 - val_acc: 0.9350 746 Epoch 786/1000 747 65s 130ms/step - loss: 0.1041 - acc: 0.9992 - val_loss: 0.3559 - val_acc: 0.9349 748 Epoch 787/1000 749 65s 130ms/step - loss: 0.1042 - acc: 0.9992 - val_loss: 0.3547 - val_acc: 0.9336 750 Epoch 788/1000 751 65s 130ms/step - loss: 0.1039 - acc: 0.9992 - val_loss: 0.3523 - val_acc: 0.9347 752 Epoch 789/1000 753 65s 130ms/step - loss: 0.1037 - acc: 0.9991 - val_loss: 0.3487 - val_acc: 0.9375 754 Epoch 790/1000 755 65s 130ms/step - loss: 0.1034 - acc: 0.9992 - val_loss: 0.3481 - val_acc: 0.9365 756 Epoch 791/1000 757 65s 130ms/step - loss: 0.1034 - acc: 0.9992 - val_loss: 0.3514 - val_acc: 0.9370 758 Epoch 792/1000 759 65s 130ms/step - loss: 0.1034 - acc: 0.9991 - val_loss: 0.3507 - val_acc: 0.9363 760 Epoch 793/1000 761 65s 130ms/step - loss: 0.1029 - acc: 0.9992 - val_loss: 0.3531 - val_acc: 0.9358 762 Epoch 794/1000 763 65s 129ms/step - loss: 0.1032 - acc: 0.9990 - val_loss: 0.3563 - val_acc: 0.9351 764 Epoch 795/1000 765 65s 129ms/step - loss: 0.1026 - acc: 0.9992 - val_loss: 0.3529 - val_acc: 0.9362 766 Epoch 796/1000 767 65s 130ms/step - loss: 0.1024 - acc: 0.9992 - val_loss: 0.3511 - val_acc: 0.9360 768 Epoch 797/1000 769 65s 130ms/step - loss: 0.1023 - acc: 0.9990 - val_loss: 0.3520 - val_acc: 0.9358 770 Epoch 798/1000 771 65s 130ms/step - loss: 0.1023 - acc: 0.9990 - val_loss: 0.3524 - val_acc: 0.9354 772 Epoch 799/1000 773 65s 130ms/step - loss: 0.1022 - acc: 0.9991 - val_loss: 0.3547 - val_acc: 0.9349 774 Epoch 800/1000 775 65s 130ms/step - loss: 0.1020 - acc: 0.9991 - val_loss: 0.3548 - val_acc: 0.9356 776 Epoch 801/1000 777 65s 129ms/step - loss: 0.1016 - acc: 0.9993 - val_loss: 0.3524 - val_acc: 0.9356 778 Epoch 802/1000 779 65s 130ms/step - loss: 0.1016 - acc: 0.9992 - val_loss: 0.3516 - val_acc: 0.9360 780 Epoch 803/1000 781 65s 130ms/step - loss: 0.1015 - acc: 0.9991 - val_loss: 0.3497 - val_acc: 0.9353 782 Epoch 804/1000 783 65s 129ms/step - loss: 0.1012 - acc: 0.9992 - val_loss: 0.3520 - val_acc: 0.9355 784 Epoch 805/1000 785 65s 130ms/step - loss: 0.1014 - acc: 0.9991 - val_loss: 0.3539 - val_acc: 0.9354 786 Epoch 806/1000 787 65s 130ms/step - loss: 0.1010 - acc: 0.9990 - val_loss: 0.3580 - val_acc: 0.9352 788 Epoch 807/1000 789 65s 130ms/step - loss: 0.1011 - acc: 0.9990 - val_loss: 0.3513 - val_acc: 0.9349 790 Epoch 808/1000 791 65s 130ms/step - loss: 0.1006 - acc: 0.9992 - val_loss: 0.3521 - val_acc: 0.9367 792 Epoch 809/1000 793 65s 130ms/step - loss: 0.1005 - acc: 0.9991 - val_loss: 0.3495 - val_acc: 0.9368 794 Epoch 810/1000 795 65s 129ms/step - loss: 0.1008 - acc: 0.9988 - val_loss: 0.3529 - val_acc: 0.9350 796 Epoch 811/1000 797 65s 129ms/step - loss: 0.1001 - acc: 0.9992 - val_loss: 0.3569 - val_acc: 0.9358 798 Epoch 812/1000 799 65s 130ms/step - loss: 0.0998 - acc: 0.9991 - val_loss: 0.3532 - val_acc: 0.9355 800 Epoch 813/1000 801 65s 129ms/step - loss: 0.0996 - acc: 0.9992 - val_loss: 0.3559 - val_acc: 0.9347 802 Epoch 814/1000 803 65s 130ms/step - loss: 0.0997 - acc: 0.9992 - val_loss: 0.3532 - val_acc: 0.9345 804 Epoch 815/1000 805 65s 130ms/step - loss: 0.0996 - acc: 0.9991 - val_loss: 0.3544 - val_acc: 0.9340 806 Epoch 816/1000 807 65s 130ms/step - loss: 0.0991 - acc: 0.9991 - val_loss: 0.3529 - val_acc: 0.9358 808 Epoch 817/1000 809 65s 130ms/step - loss: 0.0984 - acc: 0.9995 - val_loss: 0.3508 - val_acc: 0.9365 810 Epoch 818/1000 811 65s 130ms/step - loss: 0.0994 - acc: 0.9989 - val_loss: 0.3533 - val_acc: 0.9362 812 Epoch 819/1000 813 65s 129ms/step - loss: 0.0987 - acc: 0.9993 - val_loss: 0.3519 - val_acc: 0.9351 814 Epoch 820/1000 815 65s 130ms/step - loss: 0.0988 - acc: 0.9991 - val_loss: 0.3528 - val_acc: 0.9352 816 Epoch 821/1000 817 65s 130ms/step - loss: 0.0983 - acc: 0.9992 - val_loss: 0.3479 - val_acc: 0.9354 818 Epoch 822/1000 819 65s 130ms/step - loss: 0.0984 - acc: 0.9991 - val_loss: 0.3485 - val_acc: 0.9367 820 Epoch 823/1000 821 65s 130ms/step - loss: 0.0985 - acc: 0.9990 - val_loss: 0.3530 - val_acc: 0.9358 822 Epoch 824/1000 823 65s 130ms/step - loss: 0.0981 - acc: 0.9992 - val_loss: 0.3464 - val_acc: 0.9377 824 Epoch 825/1000 825 65s 130ms/step - loss: 0.0978 - acc: 0.9993 - val_loss: 0.3477 - val_acc: 0.9358 826 Epoch 826/1000 827 65s 130ms/step - loss: 0.0973 - acc: 0.9992 - val_loss: 0.3468 - val_acc: 0.9364 828 Epoch 827/1000 829 65s 130ms/step - loss: 0.0979 - acc: 0.9991 - val_loss: 0.3502 - val_acc: 0.9358 830 Epoch 828/1000 831 65s 130ms/step - loss: 0.0974 - acc: 0.9991 - val_loss: 0.3470 - val_acc: 0.9356 832 Epoch 829/1000 833 65s 130ms/step - loss: 0.0969 - acc: 0.9994 - val_loss: 0.3459 - val_acc: 0.9351 834 Epoch 830/1000 835 65s 130ms/step - loss: 0.0970 - acc: 0.9990 - val_loss: 0.3528 - val_acc: 0.9347 836 Epoch 831/1000 837 65s 130ms/step - loss: 0.0969 - acc: 0.9992 - val_loss: 0.3484 - val_acc: 0.9360 838 Epoch 832/1000 839 65s 129ms/step - loss: 0.0970 - acc: 0.9992 - val_loss: 0.3542 - val_acc: 0.9353 840 Epoch 833/1000 841 65s 130ms/step - loss: 0.0969 - acc: 0.9990 - val_loss: 0.3496 - val_acc: 0.9345 842 Epoch 834/1000 843 65s 130ms/step - loss: 0.0970 - acc: 0.9990 - val_loss: 0.3460 - val_acc: 0.9372 844 Epoch 835/1000 845 65s 129ms/step - loss: 0.0960 - acc: 0.9993 - val_loss: 0.3514 - val_acc: 0.9349 846 Epoch 836/1000 847 65s 130ms/step - loss: 0.0962 - acc: 0.9994 - val_loss: 0.3420 - val_acc: 0.9376 848 Epoch 837/1000 849 65s 130ms/step - loss: 0.0960 - acc: 0.9992 - val_loss: 0.3441 - val_acc: 0.9358 850 Epoch 838/1000 851 65s 130ms/step - loss: 0.0957 - acc: 0.9993 - val_loss: 0.3474 - val_acc: 0.9368 852 Epoch 839/1000 853 65s 130ms/step - loss: 0.0955 - acc: 0.9993 - val_loss: 0.3447 - val_acc: 0.9355 854 Epoch 840/1000 855 65s 129ms/step - loss: 0.0951 - acc: 0.9995 - val_loss: 0.3508 - val_acc: 0.9355 856 Epoch 841/1000 857 65s 130ms/step - loss: 0.0951 - acc: 0.9993 - val_loss: 0.3488 - val_acc: 0.9366 858 Epoch 842/1000 859 65s 130ms/step - loss: 0.0952 - acc: 0.9992 - val_loss: 0.3500 - val_acc: 0.9368 860 Epoch 843/1000 861 65s 129ms/step - loss: 0.0952 - acc: 0.9991 - val_loss: 0.3464 - val_acc: 0.9359 862 Epoch 844/1000 863 65s 129ms/step - loss: 0.0947 - acc: 0.9993 - val_loss: 0.3470 - val_acc: 0.9365 864 Epoch 845/1000 865 65s 129ms/step - loss: 0.0947 - acc: 0.9993 - val_loss: 0.3478 - val_acc: 0.9353 866 Epoch 846/1000 867 65s 130ms/step - loss: 0.0952 - acc: 0.9990 - val_loss: 0.3501 - val_acc: 0.9355 868 Epoch 847/1000 869 65s 130ms/step - loss: 0.0944 - acc: 0.9993 - val_loss: 0.3463 - val_acc: 0.9354 870 Epoch 848/1000 871 65s 130ms/step - loss: 0.0944 - acc: 0.9993 - val_loss: 0.3504 - val_acc: 0.9351 872 Epoch 849/1000 873 65s 130ms/step - loss: 0.0941 - acc: 0.9993 - val_loss: 0.3468 - val_acc: 0.9373 874 Epoch 850/1000 875 65s 129ms/step - loss: 0.0947 - acc: 0.9988 - val_loss: 0.3432 - val_acc: 0.9378 876 Epoch 851/1000 877 65s 129ms/step - loss: 0.0943 - acc: 0.9989 - val_loss: 0.3456 - val_acc: 0.9369 878 Epoch 852/1000 879 65s 129ms/step - loss: 0.0943 - acc: 0.9990 - val_loss: 0.3471 - val_acc: 0.9365 880 Epoch 853/1000 881 65s 130ms/step - loss: 0.0940 - acc: 0.9990 - val_loss: 0.3506 - val_acc: 0.9356 882 Epoch 854/1000 883 65s 130ms/step - loss: 0.0936 - acc: 0.9992 - val_loss: 0.3498 - val_acc: 0.9358 884 Epoch 855/1000 885 65s 130ms/step - loss: 0.0934 - acc: 0.9992 - val_loss: 0.3469 - val_acc: 0.9361 886 Epoch 856/1000 887 65s 130ms/step - loss: 0.0931 - acc: 0.9993 - val_loss: 0.3483 - val_acc: 0.9361 888 Epoch 857/1000 889 65s 130ms/step - loss: 0.0930 - acc: 0.9993 - val_loss: 0.3440 - val_acc: 0.9350 890 Epoch 858/1000 891 65s 129ms/step - loss: 0.0930 - acc: 0.9991 - val_loss: 0.3445 - val_acc: 0.9365 892 Epoch 859/1000 893 65s 130ms/step - loss: 0.0928 - acc: 0.9992 - val_loss: 0.3465 - val_acc: 0.9366 894 Epoch 860/1000 895 65s 130ms/step - loss: 0.0928 - acc: 0.9990 - val_loss: 0.3527 - val_acc: 0.9345 896 Epoch 861/1000 897 65s 129ms/step - loss: 0.0924 - acc: 0.9992 - val_loss: 0.3465 - val_acc: 0.9369 898 Epoch 862/1000 899 65s 130ms/step - loss: 0.0923 - acc: 0.9992 - val_loss: 0.3445 - val_acc: 0.9366 900 Epoch 863/1000 901 65s 130ms/step - loss: 0.0923 - acc: 0.9992 - val_loss: 0.3476 - val_acc: 0.9362 902 Epoch 864/1000 903 65s 130ms/step - loss: 0.0920 - acc: 0.9993 - val_loss: 0.3454 - val_acc: 0.9369 904 Epoch 865/1000 905 65s 130ms/step - loss: 0.0922 - acc: 0.9990 - val_loss: 0.3486 - val_acc: 0.9337 906 Epoch 866/1000 907 65s 130ms/step - loss: 0.0914 - acc: 0.9994 - val_loss: 0.3489 - val_acc: 0.9355 908 Epoch 867/1000 909 65s 129ms/step - loss: 0.0918 - acc: 0.9991 - val_loss: 0.3467 - val_acc: 0.9359 910 Epoch 868/1000 911 65s 130ms/step - loss: 0.0918 - acc: 0.9992 - val_loss: 0.3486 - val_acc: 0.9348 912 Epoch 869/1000 913 65s 130ms/step - loss: 0.0913 - acc: 0.9992 - val_loss: 0.3437 - val_acc: 0.9364 914 Epoch 870/1000 915 65s 130ms/step - loss: 0.0914 - acc: 0.9992 - val_loss: 0.3488 - val_acc: 0.9350 916 Epoch 871/1000 917 65s 130ms/step - loss: 0.0913 - acc: 0.9991 - val_loss: 0.3473 - val_acc: 0.9367 918 Epoch 872/1000 919 65s 130ms/step - loss: 0.0911 - acc: 0.9992 - val_loss: 0.3448 - val_acc: 0.9380 920 Epoch 873/1000 921 65s 130ms/step - loss: 0.0907 - acc: 0.9993 - val_loss: 0.3439 - val_acc: 0.9373 922 Epoch 874/1000 923 65s 130ms/step - loss: 0.0911 - acc: 0.9988 - val_loss: 0.3421 - val_acc: 0.9384 924 Epoch 875/1000 925 65s 130ms/step - loss: 0.0904 - acc: 0.9992 - val_loss: 0.3430 - val_acc: 0.9365 926 Epoch 876/1000 927 65s 130ms/step - loss: 0.0908 - acc: 0.9990 - val_loss: 0.3471 - val_acc: 0.9355 928 Epoch 877/1000 929 65s 129ms/step - loss: 0.0905 - acc: 0.9991 - val_loss: 0.3452 - val_acc: 0.9359 930 Epoch 878/1000 931 65s 130ms/step - loss: 0.0905 - acc: 0.9990 - val_loss: 0.3379 - val_acc: 0.9372 932 Epoch 879/1000 933 65s 130ms/step - loss: 0.0906 - acc: 0.9989 - val_loss: 0.3442 - val_acc: 0.9369 934 Epoch 880/1000 935 65s 130ms/step - loss: 0.0903 - acc: 0.9990 - val_loss: 0.3413 - val_acc: 0.9363 936 Epoch 881/1000 937 65s 130ms/step - loss: 0.0898 - acc: 0.9992 - val_loss: 0.3437 - val_acc: 0.9354 938 Epoch 882/1000 939 65s 129ms/step - loss: 0.0898 - acc: 0.9992 - val_loss: 0.3421 - val_acc: 0.9371 940 Epoch 883/1000 941 65s 130ms/step - loss: 0.0897 - acc: 0.9991 - val_loss: 0.3442 - val_acc: 0.9363 942 Epoch 884/1000 943 65s 130ms/step - loss: 0.0900 - acc: 0.9990 - val_loss: 0.3471 - val_acc: 0.9366 944 Epoch 885/1000 945 65s 130ms/step - loss: 0.0897 - acc: 0.9991 - val_loss: 0.3443 - val_acc: 0.9361 946 Epoch 886/1000 947 65s 130ms/step - loss: 0.0892 - acc: 0.9990 - val_loss: 0.3434 - val_acc: 0.9355 948 Epoch 887/1000 949 65s 130ms/step - loss: 0.0890 - acc: 0.9991 - val_loss: 0.3411 - val_acc: 0.9367 950 Epoch 888/1000 951 65s 130ms/step - loss: 0.0889 - acc: 0.9992 - val_loss: 0.3478 - val_acc: 0.9338 952 Epoch 889/1000 953 65s 130ms/step - loss: 0.0889 - acc: 0.9991 - val_loss: 0.3404 - val_acc: 0.9366 954 Epoch 890/1000 955 65s 130ms/step - loss: 0.0889 - acc: 0.9991 - val_loss: 0.3356 - val_acc: 0.9373 956 Epoch 891/1000 957 65s 130ms/step - loss: 0.0886 - acc: 0.9992 - val_loss: 0.3358 - val_acc: 0.9362 958 Epoch 892/1000 959 65s 130ms/step - loss: 0.0883 - acc: 0.9992 - val_loss: 0.3380 - val_acc: 0.9368 960 Epoch 893/1000 961 65s 129ms/step - loss: 0.0886 - acc: 0.9991 - val_loss: 0.3369 - val_acc: 0.9374 962 Epoch 894/1000 963 65s 130ms/step - loss: 0.0881 - acc: 0.9993 - val_loss: 0.3397 - val_acc: 0.9386 964 Epoch 895/1000 965 65s 130ms/step - loss: 0.0885 - acc: 0.9991 - val_loss: 0.3400 - val_acc: 0.9365 966 Epoch 896/1000 967 65s 130ms/step - loss: 0.0883 - acc: 0.9989 - val_loss: 0.3367 - val_acc: 0.9355 968 Epoch 897/1000 969 65s 130ms/step - loss: 0.0886 - acc: 0.9986 - val_loss: 0.3375 - val_acc: 0.9361 970 Epoch 898/1000 971 65s 130ms/step - loss: 0.0878 - acc: 0.9989 - val_loss: 0.3444 - val_acc: 0.9354 972 Epoch 899/1000 973 65s 130ms/step - loss: 0.0875 - acc: 0.9992 - val_loss: 0.3444 - val_acc: 0.9367 974 Epoch 900/1000 975 65s 130ms/step - loss: 0.0877 - acc: 0.9990 - val_loss: 0.3457 - val_acc: 0.9353 976 Epoch 901/1000 977 lr changed to 9.999999310821295e-05 978 66s 132ms/step - loss: 0.0873 - acc: 0.9992 - val_loss: 0.3442 - val_acc: 0.9350 979 Epoch 902/1000 980 66s 133ms/step - loss: 0.0867 - acc: 0.9994 - val_loss: 0.3425 - val_acc: 0.9361 981 Epoch 903/1000 982 66s 132ms/step - loss: 0.0874 - acc: 0.9991 - val_loss: 0.3432 - val_acc: 0.9358 983 Epoch 904/1000 984 66s 131ms/step - loss: 0.0872 - acc: 0.9992 - val_loss: 0.3431 - val_acc: 0.9360 985 Epoch 905/1000 986 66s 131ms/step - loss: 0.0871 - acc: 0.9991 - val_loss: 0.3426 - val_acc: 0.9371 987 Epoch 906/1000 988 66s 132ms/step - loss: 0.0868 - acc: 0.9991 - val_loss: 0.3422 - val_acc: 0.9371 989 Epoch 907/1000 990 66s 132ms/step - loss: 0.0869 - acc: 0.9993 - val_loss: 0.3418 - val_acc: 0.9368 991 Epoch 908/1000 992 66s 132ms/step - loss: 0.0870 - acc: 0.9991 - val_loss: 0.3415 - val_acc: 0.9366 993 Epoch 909/1000 994 66s 131ms/step - loss: 0.0863 - acc: 0.9995 - val_loss: 0.3410 - val_acc: 0.9371 995 Epoch 910/1000 996 66s 131ms/step - loss: 0.0870 - acc: 0.9991 - val_loss: 0.3405 - val_acc: 0.9363 997 Epoch 911/1000 998 66s 132ms/step - loss: 0.0864 - acc: 0.9995 - val_loss: 0.3412 - val_acc: 0.9367 999 Epoch 912/1000 1000 66s 132ms/step - loss: 0.0862 - acc: 0.9995 - val_loss: 0.3403 - val_acc: 0.9370 1001 Epoch 913/1000 1002 78s 155ms/step - loss: 0.0862 - acc: 0.9995 - val_loss: 0.3399 - val_acc: 0.9368 1003 Epoch 914/1000 1004 84s 168ms/step - loss: 0.0860 - acc: 0.9997 - val_loss: 0.3402 - val_acc: 0.9373 1005 Epoch 915/1000 1006 65s 130ms/step - loss: 0.0865 - acc: 0.9994 - val_loss: 0.3403 - val_acc: 0.9371 1007 Epoch 916/1000 1008 65s 130ms/step - loss: 0.0866 - acc: 0.9993 - val_loss: 0.3399 - val_acc: 0.9369 1009 Epoch 917/1000 1010 65s 130ms/step - loss: 0.0868 - acc: 0.9992 - val_loss: 0.3385 - val_acc: 0.9378 1011 Epoch 918/1000 1012 65s 129ms/step - loss: 0.0865 - acc: 0.9993 - val_loss: 0.3374 - val_acc: 0.9376 1013 Epoch 919/1000 1014 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3378 - val_acc: 0.9373 1015 Epoch 920/1000 1016 65s 130ms/step - loss: 0.0864 - acc: 0.9993 - val_loss: 0.3373 - val_acc: 0.9380 1017 Epoch 921/1000 1018 65s 130ms/step - loss: 0.0863 - acc: 0.9995 - val_loss: 0.3374 - val_acc: 0.9375 1019 Epoch 922/1000 1020 65s 130ms/step - loss: 0.0863 - acc: 0.9993 - val_loss: 0.3371 - val_acc: 0.9376 1021 Epoch 923/1000 1022 65s 129ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3372 - val_acc: 0.9370 1023 Epoch 924/1000 1024 65s 130ms/step - loss: 0.0860 - acc: 0.9994 - val_loss: 0.3374 - val_acc: 0.9369 1025 Epoch 925/1000 1026 65s 130ms/step - loss: 0.0860 - acc: 0.9996 - val_loss: 0.3375 - val_acc: 0.9368 1027 Epoch 926/1000 1028 65s 130ms/step - loss: 0.0862 - acc: 0.9994 - val_loss: 0.3378 - val_acc: 0.9373 1029 Epoch 927/1000 1030 65s 130ms/step - loss: 0.0864 - acc: 0.9992 - val_loss: 0.3384 - val_acc: 0.9371 1031 Epoch 928/1000 1032 65s 130ms/step - loss: 0.0863 - acc: 0.9993 - val_loss: 0.3386 - val_acc: 0.9367 1033 Epoch 929/1000 1034 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3392 - val_acc: 0.9365 1035 Epoch 930/1000 1036 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3386 - val_acc: 0.9368 1037 Epoch 931/1000 1038 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3384 - val_acc: 0.9375 1039 Epoch 932/1000 1040 65s 130ms/step - loss: 0.0856 - acc: 0.9996 - val_loss: 0.3388 - val_acc: 0.9376 1041 Epoch 933/1000 1042 65s 130ms/step - loss: 0.0859 - acc: 0.9995 - val_loss: 0.3390 - val_acc: 0.9376 1043 Epoch 934/1000 1044 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3389 - val_acc: 0.9375 1045 Epoch 935/1000 1046 65s 130ms/step - loss: 0.0859 - acc: 0.9994 - val_loss: 0.3390 - val_acc: 0.9376 1047 Epoch 936/1000 1048 65s 130ms/step - loss: 0.0859 - acc: 0.9994 - val_loss: 0.3397 - val_acc: 0.9373 1049 Epoch 937/1000 1050 65s 130ms/step - loss: 0.0858 - acc: 0.9995 - val_loss: 0.3396 - val_acc: 0.9371 1051 Epoch 938/1000 1052 65s 130ms/step - loss: 0.0860 - acc: 0.9994 - val_loss: 0.3390 - val_acc: 0.9379 1053 Epoch 939/1000 1054 65s 130ms/step - loss: 0.0859 - acc: 0.9993 - val_loss: 0.3393 - val_acc: 0.9382 1055 Epoch 940/1000 1056 65s 130ms/step - loss: 0.0858 - acc: 0.9994 - val_loss: 0.3391 - val_acc: 0.9379 1057 Epoch 941/1000 1058 65s 130ms/step - loss: 0.0858 - acc: 0.9995 - val_loss: 0.3392 - val_acc: 0.9378 1059 Epoch 942/1000 1060 65s 130ms/step - loss: 0.0857 - acc: 0.9995 - val_loss: 0.3396 - val_acc: 0.9382 1061 Epoch 943/1000 1062 65s 130ms/step - loss: 0.0858 - acc: 0.9995 - val_loss: 0.3403 - val_acc: 0.9376 1063 Epoch 944/1000 1064 65s 130ms/step - loss: 0.0859 - acc: 0.9993 - val_loss: 0.3405 - val_acc: 0.9374 1065 Epoch 945/1000 1066 65s 130ms/step - loss: 0.0854 - acc: 0.9996 - val_loss: 0.3402 - val_acc: 0.9371 1067 Epoch 946/1000 1068 65s 130ms/step - loss: 0.0859 - acc: 0.9994 - val_loss: 0.3398 - val_acc: 0.9376 1069 Epoch 947/1000 1070 65s 130ms/step - loss: 0.0857 - acc: 0.9994 - val_loss: 0.3397 - val_acc: 0.9371 1071 Epoch 948/1000 1072 65s 129ms/step - loss: 0.0855 - acc: 0.9996 - val_loss: 0.3396 - val_acc: 0.9375 1073 Epoch 949/1000 1074 65s 130ms/step - loss: 0.0853 - acc: 0.9996 - val_loss: 0.3398 - val_acc: 0.9376 1075 Epoch 950/1000 1076 65s 130ms/step - loss: 0.0856 - acc: 0.9996 - val_loss: 0.3397 - val_acc: 0.9378 1077 Epoch 951/1000 1078 65s 130ms/step - loss: 0.0856 - acc: 0.9994 - val_loss: 0.3393 - val_acc: 0.9375 1079 Epoch 952/1000 1080 65s 130ms/step - loss: 0.0857 - acc: 0.9996 - val_loss: 0.3397 - val_acc: 0.9374 1081 Epoch 953/1000 1082 65s 130ms/step - loss: 0.0854 - acc: 0.9995 - val_loss: 0.3400 - val_acc: 0.9378 1083 Epoch 954/1000 1084 65s 130ms/step - loss: 0.0856 - acc: 0.9995 - val_loss: 0.3401 - val_acc: 0.9368 1085 Epoch 955/1000 1086 65s 129ms/step - loss: 0.0855 - acc: 0.9994 - val_loss: 0.3403 - val_acc: 0.9370 1087 Epoch 956/1000 1088 65s 130ms/step - loss: 0.0856 - acc: 0.9994 - val_loss: 0.3405 - val_acc: 0.9371 1089 Epoch 957/1000 1090 65s 130ms/step - loss: 0.0857 - acc: 0.9994 - val_loss: 0.3408 - val_acc: 0.9375 1091 Epoch 958/1000 1092 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3405 - val_acc: 0.9374 1093 Epoch 959/1000 1094 65s 130ms/step - loss: 0.0856 - acc: 0.9993 - val_loss: 0.3408 - val_acc: 0.9375 1095 Epoch 960/1000 1096 65s 130ms/step - loss: 0.0856 - acc: 0.9995 - val_loss: 0.3407 - val_acc: 0.9369 1097 Epoch 961/1000 1098 65s 130ms/step - loss: 0.0853 - acc: 0.9996 - val_loss: 0.3402 - val_acc: 0.9371 1099 Epoch 962/1000 1100 65s 129ms/step - loss: 0.0855 - acc: 0.9994 - val_loss: 0.3399 - val_acc: 0.9371 1101 Epoch 963/1000 1102 65s 129ms/step - loss: 0.0852 - acc: 0.9996 - val_loss: 0.3400 - val_acc: 0.9378 1103 Epoch 964/1000 1104 65s 129ms/step - loss: 0.0856 - acc: 0.9994 - val_loss: 0.3399 - val_acc: 0.9375 1105 Epoch 965/1000 1106 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3396 - val_acc: 0.9375 1107 Epoch 966/1000 1108 65s 130ms/step - loss: 0.0852 - acc: 0.9995 - val_loss: 0.3391 - val_acc: 0.9368 1109 Epoch 967/1000 1110 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3383 - val_acc: 0.9374 1111 Epoch 968/1000 1112 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3384 - val_acc: 0.9375 1113 Epoch 969/1000 1114 65s 130ms/step - loss: 0.0851 - acc: 0.9997 - val_loss: 0.3383 - val_acc: 0.9375 1115 Epoch 970/1000 1116 65s 129ms/step - loss: 0.0853 - acc: 0.9995 - val_loss: 0.3388 - val_acc: 0.9365 1117 Epoch 971/1000 1118 65s 130ms/step - loss: 0.0851 - acc: 0.9996 - val_loss: 0.3381 - val_acc: 0.9356 1119 Epoch 972/1000 1120 65s 130ms/step - loss: 0.0855 - acc: 0.9994 - val_loss: 0.3387 - val_acc: 0.9362 1121 Epoch 973/1000 1122 65s 130ms/step - loss: 0.0857 - acc: 0.9994 - val_loss: 0.3385 - val_acc: 0.9372 1123 Epoch 974/1000 1124 65s 130ms/step - loss: 0.0854 - acc: 0.9995 - val_loss: 0.3385 - val_acc: 0.9373 1125 Epoch 975/1000 1126 65s 130ms/step - loss: 0.0851 - acc: 0.9996 - val_loss: 0.3380 - val_acc: 0.9375 1127 Epoch 976/1000 1128 65s 130ms/step - loss: 0.0853 - acc: 0.9994 - val_loss: 0.3380 - val_acc: 0.9379 1129 Epoch 977/1000 1130 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3374 - val_acc: 0.9376 1131 Epoch 978/1000 1132 65s 130ms/step - loss: 0.0851 - acc: 0.9996 - val_loss: 0.3376 - val_acc: 0.9379 1133 Epoch 979/1000 1134 65s 130ms/step - loss: 0.0853 - acc: 0.9995 - val_loss: 0.3380 - val_acc: 0.9378 1135 Epoch 980/1000 1136 65s 130ms/step - loss: 0.0852 - acc: 0.9995 - val_loss: 0.3376 - val_acc: 0.9381 1137 Epoch 981/1000 1138 65s 130ms/step - loss: 0.0854 - acc: 0.9995 - val_loss: 0.3377 - val_acc: 0.9381 1139 Epoch 982/1000 1140 65s 129ms/step - loss: 0.0849 - acc: 0.9996 - val_loss: 0.3373 - val_acc: 0.9384 1141 Epoch 983/1000 1142 65s 130ms/step - loss: 0.0852 - acc: 0.9993 - val_loss: 0.3372 - val_acc: 0.9379 1143 Epoch 984/1000 1144 65s 130ms/step - loss: 0.0848 - acc: 0.9997 - val_loss: 0.3368 - val_acc: 0.9381 1145 Epoch 985/1000 1146 65s 130ms/step - loss: 0.0852 - acc: 0.9994 - val_loss: 0.3373 - val_acc: 0.9382 1147 Epoch 986/1000 1148 65s 130ms/step - loss: 0.0847 - acc: 0.9997 - val_loss: 0.3372 - val_acc: 0.9380 1149 Epoch 987/1000 1150 65s 130ms/step - loss: 0.0848 - acc: 0.9996 - val_loss: 0.3371 - val_acc: 0.9387 1151 Epoch 988/1000 1152 65s 130ms/step - loss: 0.0853 - acc: 0.9993 - val_loss: 0.3377 - val_acc: 0.9380 1153 Epoch 989/1000 1154 65s 130ms/step - loss: 0.0851 - acc: 0.9995 - val_loss: 0.3371 - val_acc: 0.9385 1155 Epoch 990/1000 1156 65s 130ms/step - loss: 0.0848 - acc: 0.9996 - val_loss: 0.3379 - val_acc: 0.9384 1157 Epoch 991/1000 1158 65s 130ms/step - loss: 0.0849 - acc: 0.9994 - val_loss: 0.3377 - val_acc: 0.9380 1159 Epoch 992/1000 1160 65s 130ms/step - loss: 0.0852 - acc: 0.9993 - val_loss: 0.3370 - val_acc: 0.9381 1161 Epoch 993/1000 1162 65s 130ms/step - loss: 0.0851 - acc: 0.9994 - val_loss: 0.3371 - val_acc: 0.9380 1163 Epoch 994/1000 1164 65s 130ms/step - loss: 0.0848 - acc: 0.9996 - val_loss: 0.3371 - val_acc: 0.9381 1165 Epoch 995/1000 1166 65s 130ms/step - loss: 0.0849 - acc: 0.9994 - val_loss: 0.3381 - val_acc: 0.9381 1167 Epoch 996/1000 1168 65s 130ms/step - loss: 0.0849 - acc: 0.9996 - val_loss: 0.3379 - val_acc: 0.9379 1169 Epoch 997/1000 1170 65s 130ms/step - loss: 0.0853 - acc: 0.9993 - val_loss: 0.3384 - val_acc: 0.9377 1171 Epoch 998/1000 1172 65s 129ms/step - loss: 0.0849 - acc: 0.9995 - val_loss: 0.3393 - val_acc: 0.9369 1173 Epoch 999/1000 1174 65s 130ms/step - loss: 0.0849 - acc: 0.9994 - val_loss: 0.3395 - val_acc: 0.9369 1175 Epoch 1000/1000 1176 65s 130ms/step - loss: 0.0847 - acc: 0.9996 - val_loss: 0.3389 - val_acc: 0.9371 1177 Train loss: 0.08910960255563259 1178 Train accuracy: 0.9977200021743774 1179 Test loss: 0.3388938118517399 1180 Test accuracy: 0.9371000009775162
测试准确率到了93.71%，比之前的都高一点。
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458