上一篇是基础的数据输入,然而继续查看模型的输入数据会遇到各种数据结构,比如迭代器yield
本文将继续寻根究底,看看是如何输入audio及rgb 特征,然后得到NeXtVLAD聚类后的特征。
我看了下模型的输出,似乎只能输出类别,而我想得到某一层的输出结果。这个咋整呢??
屁大点事结果我麻烦了百度的很多人帮我解决,真是辛苦了,从此再也不说飞浆的坏话了,哈哈。
我觉得杀鸡就得用牛刀,快!自己瞎摸索半天甚至更久真的是浪费时间。
以后有问题我还是会让全世界的人都知道,感谢所有帮助我的大佬。
下面直接看模型的该层输出,以及模型的加载方法。难道说没有GPU就不行吗???
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 783, in run
six.reraise(*sys.exc_info())
File "D:\python36\lib\site-packages\six.py", line 703, in reraise
raise value
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 778, in run
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 831, in _run_impl
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 851, in _run_program
assert len(feed) == 1, "Not compiled with data parallel"
AssertionError: Not compiled with data parallel
但我在服务器用GPU也是得到同样的结果,卧槽,这是为啥子,后来一想没有变成飞浆的tensor,我改成tensor呢?
改成了飞浆的tensor也不行,必须用下面的格式输入,卧槽,这就奇怪了。下面老子解剖下这个函数到底是啥玩意。
feed=infer_feeder.feed(data_feed_in))
解读下面的我以为是套个列表即可:
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,data.shape)
Traceback (most recent call last):
File "<pyshell#2>", line 2, in <module>
print(infer_iter,data.shape)
AttributeError: 'list' object has no attribute 'shape'
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,len(data))
0 1
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,len(data[0]))
0 2
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,data[0][0].shape[0])
0 3
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,data[0][0].shape)
0 (3, 1024)
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,data[0][1].shape)
0 (3, 128)
>>> data_feed_in = [items for items in data]
>>> data_feed_in==data
True
后来发现还是有infer_feeder的问题:
Warning (from warnings module):
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 782
"The following exception is not an EOF exception.")
UserWarning: The following exception is not an EOF exception.
Traceback (most recent call last):
File "D:/python36/new/rcdnn/image/my_predict_2.py", line 123, in <module>
feed=[[image_feature,audio_feature]])#feed=infer_feeder.feed(data_feed_in))
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 783, in run
six.reraise(*sys.exc_info())
File "D:\python36\lib\site-packages\six.py", line 703, in reraise
raise value
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 778, in run
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 831, in _run_impl
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 857, in _run_program
(type(feed)))
TypeError: feed requires dict as its Parameter. But you passed in <class 'list'>
人家说要dict,我给的列表不对,哦对啊,应该是字典才对,试试
>>> infer_feeder.feed(data)
{'test_rgb': <paddle.fluid.core_avx.LoDTensor object at 0x000002D8B6262068>, 'test_audio': <paddle.fluid.core_avx.LoDTensor object at 0x000002D8B625DFB8>}
但下面还是不对,我去,非要用infer_feeder啊??
Warning (from warnings module):
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 782
"The following exception is not an EOF exception.")
UserWarning: The following exception is not an EOF exception.
Traceback (most recent call last):
File "D:/python36/new/rcdnn/image/my_predict_2.py", line 123, in <module>
feed={'test_rgb':image_feature,'test_audio':audio_feature})#feed=infer_feeder.feed(data_feed_in))
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 783, in run
six.reraise(*sys.exc_info())
File "D:\python36\lib\site-packages\six.py", line 703, in reraise
raise value
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 778, in run
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 831, in _run_impl
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 905, in _run_program
fetch_var_name)
paddle.fluid.core_avx.EnforceNotMet:
--------------------------------------------
C++ Call Stacks (More useful to developers):
--------------------------------------------
Windows not support stack backtrace yet.
------------------------------------------
Python Call Stacks (More useful to users):
------------------------------------------
File "D:\python36\lib\site-packages\paddle\fluid\framework.py", line 2525, in append_op
attrs=kwargs.get("attrs", None))
File "D:\python36\lib\site-packages\paddle\fluid\layer_helper.py", line 43, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)
File "D:\python36\lib\site-packages\paddle\fluid\layers\sequence_lod.py", line 350, in sequence_pool
"pad_value": pad_value
File "D:/python36/new/rcdnn/image\models\nextvlad\nextvlad_model.py", line 76, in forward
a_sum = fluid.layers.sequence_pool(activation, 'sum')
File "D:/python36/new/rcdnn/image\models\nextvlad\nextvlad_model.py", line 162, in create_model
vlad_video = video_nextvlad.forward(video_input)
File "D:/python36/new/rcdnn/image\models\nextvlad\nextvlad.py", line 117, in build_model
rgb, audio, is_training=(self.mode == 'train'), **model_args)
File "D:/python36/new/rcdnn/image/my_predict_2.py", line 84, in <module>
infer_model.build_model()
File "D:\python36\lib\idlelib\run.py", line 474, in runcode
exec(code, self.locals)
File "D:\python36\lib\idlelib\run.py", line 144, in main
ret = method(*args, **kwargs)
File "<string>", line 1, in <module>
----------------------
Error Message Summary:
----------------------
InvalidArgumentError: Input(X) Tensor of SequencePoolOp does not contain LoD information.
[Hint: Expected lod_level > 0, but received lod_level:0 <= 0:0.] at (D:\1.7.1\paddle\paddle/fluid/operators/sequence_ops/sequence_pool_op.h:43)
[operator < sequence_pool > error]
加个[]也是不行,
RuntimeError: Some of your feed data hold LoD information.
They can not be completely cast from a list of Python
ndarray to LoDTensor. Please convert data to LoDTensor
directly before feeding the data.
人家说了要改成飞浆的tensor,我试试numpy咋改成LoDTensor??结果仍旧不行,百度的大佬说yield返回numpy即可。
但是寡人看了即使有没有yield不重要,但必须有data_feeder,这是标准化的投喂数据,不能少,这一点还没tf的方便。
官方例子:
| import numpy as np
| import paddle.fluid as fluid
|
| def generate_reader(batch_size, base=0, factor=1):
| def _reader():
| for i in range(batch_size):
| yield np.ones([4]) * factor + base, np.ones([4]) * factor + base + 5
| return _reader()
|
| x = fluid.data(name='x', shape=[None, 2, 2])
| y = fluid.data(name='y', shape=[None, 2, 2], dtype='float32')
|
| z = fluid.layers.elementwise_add(x, y)
|
| feeder = fluid.DataFeeder(['x','y'], fluid.CPUPlace())
| place_num = 2
| places = [fluid.CPUPlace() for x in range(place_num)]
| data = []
| exe = fluid.Executor(fluid.CPUPlace())
| exe.run(fluid.default_startup_program())
| program = fluid.CompiledProgram(fluid.default_main_program()).with_data_parallel(places=places)
|
| # print sample feed_parallel r result
| # for item in list(feeder.feed_parallel([generate_reader(5, 0, 1), generate_reader(3, 10, 2)], 2)):
| # print(item['x'])
| # print(item['y'])
|
| reader_list = [generate_reader(5, 0, 1), generate_reader(3, 10, 2)]
| res = exe.run(program=program, feed=list(feeder.feed_parallel(reader_list, 2)), fetch_list=[z])
| print(res)
因此我也不改了,就这样吧,准备写文档。
另外有相关问题可以加入QQ群讨论,不设微信群
QQ群:868373192
语音图像视频深度-学习群