报错解决:ResourceExhaustedError: OOM when allocating tensor with shape
早上在使用tensorflow时遇到如下报错:
Traceback (most recent call last):
File "C:\Users\peter\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
execfile(filename, namespace)
File "C:\Users\peter\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
callbacks = [early_stopping,cp_callback]) # pass callback to training
File "C:\Users\peter\Anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py", line 880, in fit
validation_steps=validation_steps)
File "C:\Users\peter\Anaconda3\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 266, in model_iteration
batch_outs = f(actual_inputs)
File "C:\Users\peter\Anaconda3\lib\site-packages\tensorflow\python\keras\backend.py", line 3076, in __call__
run_metadata=self.run_metadata)
File "C:\Users\peter\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1439, in __call__
run_metadata_ptr)
File "C:\Users\peter\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 528, in __exit__
c_api.TF_GetCode(self.status.status))
ResourceExhaustedError: OOM when allocating tensor with shape[180621,64,1,1981] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node conv1d/conv1d/Conv2D}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
这个报错的意思是你的数据太多了,不能一次放进GPU中,虽然你设置了batch_size,但tensorflow默认是一次把所有数据都放进GPU中。
我的解决方案是采用CPU训练
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"