在网络连接良好的情况下训练网络,训练的时候突然出现ConnectionRefusedError: [Errno 111] Connection refused,而且此时第一个Epoch还没有运行完成!
有可能是训练数据出了问题。比如,如下报错信息:
Traceback (most recent call last):
File "train.py", line 90, in <module>
train(opt, data_loader, model, visualizer)
File "train.py", line 34, in train
for i, data in enumerate(dataset):
File "/********/anaconda3/envs/python36/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 267, in __next__
return self._process_next_batch(batch)
File "/********/anaconda3/envs/python36/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 301, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
OSError: Traceback (most recent call last):
File "/********/anaconda3/envs/python36/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 55, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/********/anaconda3/envs/python36/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 55, in <listcomp>
samples = collate_fn([dataset[i] for i in batch_indices])
File "/********/DeepLearningLCT/DeblurGAN-master/self_data_multitask_1024/data/aligned_dataset.py", line 31, in __getitem__
AB = Image.open(AB_path).convert('L')
File "/********/anaconda3/envs/python36/lib/python3.6/site-packages/PIL/Image.py", line 2687, in open
% (filename if filename else fp))
OSError: cannot identify image file './data/data_1024/AB/train/31636_7.png'
Exception ignored in: <bound method DataLoaderIter.__del__ of <torch.utils.data.dataloader.DataLoaderIter object at 0x7fdfd12f43c8>>
Traceback (most recent call last):
File "/********/anaconda3/envs/python36/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 333, in __del__
self._shutdown_workers()
File "/********/anaconda3/envs/python36/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 319, in _shutdown_workers
self.data_queue.get()
File "/********anaconda3/envs/python36/lib/python3.6/multiprocessing/queues.py", line 337, in get
return _ForkingPickler.loads(res)
File "/********/anaconda3/envs/python36/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 70, in rebuild_storage_fd
fd = df.detach()
File "/********/anaconda3/envs/python36/lib/python3.6/multiprocessing/resource_sharer.py", line 57, in detach
with _resource_sharer.get_connection(self._id) as conn:
File "/********/anaconda3/envs/python36/lib/python3.6/multiprocessing/resource_sharer.py", line 87, in get_connection
c = Client(address, authkey=process.current_process().authkey)
File "/********/anaconda3/envs/python36/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/********/anaconda3/envs/python36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
问题并不出在服务器的网络连接上,而是出在训练集中的 './data/data_1024/AB/train/31636_7.png'上。
例如:
生成该图像的代码错误中止导致该图像无法正常读取。
解决方法:
重新生成训练数据。
成功解决