Recently I learned an ultra-lightweight super-resolution reconstruction network SESR, and the effect is not bad.
Table of contents
- 1. Source code package
- 2. Preparation of data set
- 3. Training environment configuration
- 4. Training
- 5. Quantitative training
- 6. Model inference test
- 7. Super score effect
- 8. Summary
1. Source code package
The address of SESR official website is: official website
The source code package address that I adjusted myself is: SESR complete package extraction code: b80m
Paper address: paper
It is recommended to use the source code package provided by me. I have commented many places and it seems effortless. I also added the inference test script myself.
After downloading the source code package and decompressing it, it will look like this:
2. Preparation of data set
There are two ways to obtain the data set:
2.1 Official website download
Directly running the script file train.py in the source code package will automatically download the div2k data set first, but the download is very slow. The high-resolution data set is more than 3G, and it is easy to download. By default, it will be downloaded to the system C drive. The specific path is:
C:\Users\Administrator\tensorflow_datasets\downloads. Every time the download fails, if you run it again, the serial code will be regenerated and downloaded again, which is very troublesome.
as follows:
2.2 Network disk download
I have provided a data set that I have downloaded and organized. I have organized the file storage correspondence. Scholars can directly download and import it. The download link is: network disk download, and the extraction code is: 32d4.
3. Training environment configuration
The network structure is run under the TensorFlow framework. The TensorFlow version is 2.3. There is also a package version of tensorflow_datasets==4.1, Pyhton version 3.6, eh. . . . . . . . . . . . . . . . . .
After going through a lot of pitfalls, the version I finally adjusted myself was TensorFlow-gpu.2.9, Python 3.7 version, tensorflow_datasets4.8.2, as follows:
After installing TensorFlow-GPU, first test whether the GPU can be called normally. Refer to the test method: Add link description
4. Training
4.1 Modify configuration parameters
Open the train.py file and modify some configuration parameters according to your computer conditions:
The modifications in the train.py script corresponding to the above picture are as follows:
4.2 Import data set
After downloading the data set I provided, unzip the entire tensorflow_datasets folder and place it in the data folder, and assign the path to the tensorflow_datasets folder to the variable data_dir. The specific modifications in the code are as follows:
4.3 2x super-resolution network training
Choose the training depth according to your needs:
4.3.1 Training SESR-M5 network
where m = 5, f = 16, feature_size = 256, with folded linear blocks:
python train.py
4.3.2 Training SESR-M5 network
m = 5, f = 16, feature_size = 256, extended linear block:
python train.py --linear_block_type expanded
4.3.3 Training SESR-M11 network
where m = 11, f = 16, feature_size = 64, with folded linear blocks:
python train.py --m 11 --feature_size 64
4.4.4 Training SESR-XL network
where m = 11, f = 16, feature_size = 64, with folded linear blocks:
python train.py --m 11 --int_features 32 --feature_size 64
4.4 2x super-resolution network model
After training through the above steps, the weight files and models will be automatically saved in the logs file. The weight files of the models I trained myself are packaged in the source code package, and scholars can use them directly, as follows:
The contents represented by each of the above files are:
.pb: Represents protocol buffers, which is a binary serialized file of model structure and parameters. Stores the network structure, variables, weights and other information of the model. Is the main file of the model persist.
.data-00000-of-00001: stores the value of the model variable, that is, the value of the model weight parameter. The weights saved after model training is completed.
.index: Index file, which stores the meta information of the parameter tensor, such as tensor name, dimensions, etc. Used to locate tensor data in the data file.
checkpoints file: stores parameters during model training and is used to resume training.
4.5 Modify model saving format
The above is the default saving method. If seniors need to modify the saving method in other formats, the specific modifications are as follows:
4.6 4x super-resolution network training
The 4x super-resolution network must be trained on the basis of the 2x super-resolution model. You can choose the network depth yourself:
4.6.1 Training SESR-M5 network
where m = 5, f = 16, feature_size = 256, with folded linear blocks:
python train.py --scale 4
4.6.2 Training SESR-M5 network
m = 5, f = 16, feature_size = 256, extended linear block:
python train.py --linear_block_type expanded --scale 4
4.6.3 Training SESR-M11 network
where m = 11, f = 16, feature_size = 64, with folded linear blocks:
python train.py --m 11 --feature_size 64 --scale 4
4.6.4 Training SESR-XL network
where m = 11, f = 16, feature_size = 64, with folded linear blocks:
python train.py --m 11 --int_features 32 --feature_size 64 --scale 4
4.7 4x super-resolution network model
After training, the model will be automatically saved in the logs file, as follows:
5. Quantitative training
Run the following commands to debug the network during training and generate TFLITE (for x2 SISR, SESR-M5 networks):
python train.py --quant_W --quant_A --gen_tflite
5.1 Quantitative training model
After training, it is automatically saved in the logs/x2_models file, as follows:
6. Model inference test
The inference script is written by myself. The specific usage is as follows. You can choose according to your needs:
7. Super score effect
8. Summary
The above is a detailed graphic tutorial on super-resolution reconstruction - SESR network training and inference testing. It is not easy to summarize. Please give me a lot of support. Thank you! Welcome to leave a message for discussion.