Kaggle machine learning is the world's most famous race of artificial intelligence competition, each contestant project attracts a large number of AI enthusiasts to participate.
Pick salt deposition zone 2018 Competition for identification as an example: https://www.kaggle.com/c/tgs-salt-identification-challenge
First, the data
Kaggle can be downloaded from the site, but need to register, download speeds will likely be slow. I can be downloaded directly from the Baidu network disk:
Links: https://pan.baidu.com/s/1htvnrwQagOXHXfjpaGedPQ
extraction code: a0zx
Two, unet ++ open source model
https://github.com/MrGiovanni/UNetPlusPlus
Third, data processing and preparation
Import packages:
import os import random import matplotlib.pyplot as plt import pandas as pd import numpy as np from sklearn.model_selection import train_test_split from skimage.transform import resize from UNetPlusPlus_master.segmentation_models import Xnet from keras.preprocessing.image import load_img from keras.optimizers import * from keras.callbacks import EarlyStopping, ModelCheckpoint
Data location, size:
root = r'E:\Kaggle\salt\competition_data' model_path = root + '/model' imgs_path = root + r'\train' test_imgs_path = root + r'\test' train_csv = root + r'\train.csv' depths_csv = root + r'\depths.csv' orig_img_w = 101 orig_img_h = 101 train_img_w = 224 train_img_h = 224
Original size, and size conversion training:
def orig2tain(img): return resize(img, (train_img_w, train_img_h), mode='constant', preserve_range=True) def train2orig(img): return resize(img, (orig_img_w, orig_img_h), mode='constant', preserve_range=True)
Read data:
train_df = pd.read_csv(train_csv, usecols=[0], index_col='id') train_df["images"] = [np.array(load_img("{}/images/{}.png".format(imgs_path, idx), grayscale=False)) / 255 for idx in train_df.index] train_df["masks"] = [np.array(load_img("{}/masks/{}.png".format(imgs_path, idx), grayscale=True)) / 255 for idx in train_df.index]
Display reads the results:
max_images = 10 grid_width = 10 grid_height = int(max_images / grid_width) + 1 fig, axs = plt.subplots(grid_height, grid_width, figsize=(20, 4)) for i, idx in enumerate(train_df.index[:max_images]): img = train_df.loc[idx].images mask = train_df.loc[idx].masks ax = axs[int(i / grid_width), i % grid_width] ax.imshow(img, cmap="Greys") ax = axs[int(i / grid_width)+1, i % grid_width] ax.imshow(mask, cmap="Greens") ax.set_yticklabels([]) ax.set_xticklabels([]) plt.show()
2: 8 randomized to the training set, validation set:
train_ids, valid_ids, train_x, valid_x, train_y, valid_y = train_test_split( train_df.index.values, np.array(train_df.images.map(orig2tain).tolist()).reshape(-1, train_img_w, train_img_h, 3), np.array(train_df.masks.map(orig2tain).tolist()).reshape(-1, train_img_w, train_img_h, 1), test_size=0.2, random_state=123)
Fourth, training
input_size = (train_img_w, train_img_h, 3) model = Xnet(input_shape=input_size, backbone_name='resnet50', encoder_weights='imagenet', decoder_block_type='transpose') model.compile(optimizer = Adam(lr = 1e-4), loss = 'binary_crossentropy', metrics = ['accuracy']) model_name = 'Kaggle_Salt_{epoch:02d}-{val_acc:.3f}.hdf5' abs_model_name = os.path.join(model_path, model_name) model_checkpoint = ModelCheckpoint(abs_model_name, monitor='val_loss', verbose=2, save_best_only=True) early_stop = EarlyStopping(monitor='val_loss', patience=6) callbacks = [early_stop, model_checkpoint]
history = model.fit(train_x, train_y, validation_data=[valid_x, valid_y], epochs=100, batch_size=4, callbacks=callbacks)
Show training curve:
acc = history.history['acc'] val_acc = history.history['val_acc'] loss = history.history['loss'] val_loss = history.history['val_loss'] epochs = range(len(acc)) plt.plot(epochs, acc, 'bo', label='Training acc') plt.plot(epochs, val_acc, 'b', label='Validation acc') plt.title('Training and validation accuracy') plt.legend () plt.figure() plt.plot(epochs, loss, 'bo', label='Training loss') plt.plot(epochs, val_loss, 'b', label='Validation loss') plt.title('Training and validation loss') plt.legend () plt.show()