Gstreamer自定义Plugin及调用自定义Plugin
Gstreamer支持开发者自己创建Plugin，创建后的Plugin可以通过工具gst-inspect-1.0查看，并在代码中调用自定义的plugin。
Gstreamer 官网中给出了Plugin创建教程，但实际上如果按照教程一步步走，最后会因编译失败的问题无法编译出Plugin的lib库（至少目前在Ubuntu20.04是这样）
官网Plugin教程地址：
https://gstreamer.freedesktop.org/documentation/plugin-development/basics/boiler.html?gi-language=c
这里结合Gstreamer 官网的教程，记录以下内容：
- 自定义Gstreamer Plugin
- 通过工具（gst-inspect-1.0）查看自定义Plugin信息
- 调用自定义的Plugin
系统：Ubuntu
Gstream版本：1.0
可以通过下述命令在Ubuntu上安装Gstreamer开发环境
apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio
编译的时候，通过pkg可以直接查询Gstreamer的头文件、lib等参数。
gcc xxxxx `pkg-config --cflags --libs gstreamer-1.0`
关于Gstreamer的安装，官网给出了很详细的教程，参考官网即可。
自定义Gstreamer Plugin
下载Gstreamer提供的Plugin创建工具。
git clone https://gitlab.freedesktop.org/gstreamer/gst-template.git
下载完之后目录大概是这样的。gst-plugin/tools/目录下的make_element就是用于创建Plugin的工具。
Gsteamer通过Plugin的方式，集成了大量的Element，每个Element提供了特定的功能，将这些Element连接在一起组成了Pipleline。
目前如果直接使用make_element，并使用gstreamer1.0版本编译生成的工具生成的源文件是编译不过的。所以通过git，将gst-template这个仓库切到特定提交。
我切到到了这个Commit。
然后执行make_element创建自定义的Plugin
cd gst-template/gst-plugin/src
../tools/make_element MyTestFilter
执行后，在“gst-template/gst-plugin/src”这个目下多出来两个文件 gstmytestfliter.h 和gstmytestfliter.c。这个两个就是MyTestFilter这个Plugin，对应的源码实现。简单分析一下源码。
gstmytestfliter.h这个头文件中，需要手动修改一处内容，否会会编译不过。
/*
* GStreamer
* Copyright (C) 2005 Thomas Vander Stichele <[email protected]>
* Copyright (C) 2005 Ronald S. Bultje <[email protected]>
* Copyright (C) 2020 Niels De Graef <[email protected]>
* Copyright (C) 2023 <<[email protected]>>
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
* DEALINGS IN THE SOFTWARE.
*
* Alternatively, the contents of this file may be used under the
* GNU Lesser General Public License Version 2.1 (the "LGPL"), in
* which case the following provisions apply instead of the ones
* mentioned above:
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Library General Public License for more details.
*
* You should have received a copy of the GNU Library General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
* Boston, MA 02111-1307, USA.
*/
#ifndef __GST_MYTESTFLITER_H__
#define __GST_MYTESTFLITER_H__
#include <gst/gst.h>
G_BEGIN_DECLS
#define GST_TYPE_MYTESTFLITER (gst_my_test_fliter_get_type())
G_DECLARE_FINAL_TYPE (GstMyTestFliter, gst_my_test_fliter,
GST, MYTESTFLITER, GstElement)
// 注意：
// 上一行底代码中的MYTESTFLITER是手动修改后的
// 工具生成的值是PLUGIN_TEMPLATE，如果不修改，编译时会失败（找不到GST_MYTESTFLITER这个定义）
struct _GstMyTestFliter
{
GstElement element;
GstPad *sinkpad, *srcpad;
gboolean silent;
};
G_END_DECLS
#endif /* __GST_MYTESTFLITER_H__ */
gstmytestfliter.c：源码中主要关注两个函数，gst_my_test_fliter_class_init 用于初始化PluginClass（类似C++类的构造函数）； gst_my_test_fliter_init 用于初始化Element，当Plugin被创建时会调用这个函数。
/*
* GStreamer
* Copyright (C) 2005 Thomas Vander Stichele <[email protected]>
* Copyright (C) 2005 Ronald S. Bultje <[email protected]>
* Copyright (C) 2023 <<[email protected]>>
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
* DEALINGS IN THE SOFTWARE.
*
* Alternatively, the contents of this file may be used under the
* GNU Lesser General Public License Version 2.1 (the "LGPL"), in
* which case the following provisions apply instead of the ones
* mentioned above:
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Library General Public License for more details.
*
* You should have received a copy of the GNU Library General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
* Boston, MA 02111-1307, USA.
*/
/**
* SECTION:element-mytestfliter
*
* FIXME:Describe mytestfliter here.
*
* <refsect2>
* <title>Example launch line</title>
* |[
* gst-launch -v -m fakesrc ! mytestfliter ! fakesink silent=TRUE
* ]|
* </refsect2>
*/
#ifdef HAVE_CONFIG_H
# include <config.h>
#endif
#include <gst/gst.h>
#include "gstmytestfliter.h"
GST_DEBUG_CATEGORY_STATIC (gst_my_test_fliter_debug);
#define GST_CAT_DEFAULT gst_my_test_fliter_debug
/* Filter signals and args */
enum
{
/* FILL ME */
LAST_SIGNAL
};
enum
{
PROP_0,
PROP_SILENT
};
/* the capabilities of the inputs and outputs.
*
* describe the real formats here.
*/
static GstStaticPadTemplate sink_factory = GST_STATIC_PAD_TEMPLATE ("sink",
GST_PAD_SINK, // 类型为sink
GST_PAD_ALWAYS, // 这个PAD总是存在，Plugin创建时就自动创建了
GST_STATIC_CAPS ("ANY") // 能力值（相当于过滤器，只有能力匹配的PAD才能连接上）
);
static GstStaticPadTemplate src_factory = GST_STATIC_PAD_TEMPLATE ("src",
GST_PAD_SRC, // 类型为src
GST_PAD_ALWAYS,
GST_STATIC_CAPS ("ANY")
);
#define gst_my_test_fliter_parent_class parent_class
G_DEFINE_TYPE (GstMyTestFliter, gst_my_test_fliter, GST_TYPE_ELEMENT);
static void gst_my_test_fliter_set_property (GObject * object, guint prop_id,
const GValue * value, GParamSpec * pspec);
static void gst_my_test_fliter_get_property (GObject * object, guint prop_id,
GValue * value, GParamSpec * pspec);
static gboolean gst_my_test_fliter_sink_event (GstPad * pad, GstObject * parent, GstEvent * event);
static GstFlowReturn gst_my_test_fliter_chain (GstPad * pad, GstObject * parent, GstBuffer * buf);
/* GObject vmethod implementations */
// 初始化函数，用于初始化plugin class （相当于C++类的构造函数）
/* initialize the mytestfliter's class */
static void
gst_my_test_fliter_class_init (GstMyTestFliterClass * klass)
{
GObjectClass *gobject_class;
GstElementClass *gstelement_class;
gobject_class = (GObjectClass *) klass;
gstelement_class = (GstElementClass *) klass;
gobject_class->set_property = gst_my_test_fliter_set_property;
gobject_class->get_property = gst_my_test_fliter_get_property;
g_object_class_install_property (gobject_class, PROP_SILENT,
g_param_spec_boolean ("silent", "Silent", "Produce verbose output ?",
FALSE, G_PARAM_READWRITE));
// Plugin的描述内容
gst_element_class_set_details_simple(gstelement_class,
"MyTestFliter",
"FIXME:Generic",
"FIXME:Generic Template Element",
" <<[email protected]>>");
// 注册pad templates，注册后可以使用gst_pad_new_from_static_template创建这个pad
gst_element_class_add_pad_template (gstelement_class,
gst_static_pad_template_get (&src_factory));
// 注册pad templates，注册后可以使用gst_pad_new_from_static_template创建这个pad
gst_element_class_add_pad_template (gstelement_class,
gst_static_pad_template_get (&sink_factory));
}
// 当这个插件被创建时，会调用这个初始化函数
/* initialize the new element
* instantiate pads and add them to element
* set pad calback functions
* initialize instance structure
*/
static void
gst_my_test_fliter_init (GstMyTestFliter * filter)
{
// 创建sinkpad
filter->sinkpad = gst_pad_new_from_static_template (&sink_factory, "sink");
// 设置处理sink event的函数
gst_pad_set_event_function (filter->sinkpad,
GST_DEBUG_FUNCPTR(gst_my_test_fliter_sink_event));
// 设置sink对应的chain function（处理给到sink数据的函数）
gst_pad_set_chain_function (filter->sinkpad,
GST_DEBUG_FUNCPTR(gst_my_test_fliter_chain));
GST_PAD_SET_PROXY_CAPS (filter->sinkpad);
gst_element_add_pad (GST_ELEMENT (filter), filter->sinkpad);
filter->srcpad = gst_pad_new_from_static_template (&src_factory, "src");
GST_PAD_SET_PROXY_CAPS (filter->srcpad);
gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
filter->silent = FALSE;
}
static void
gst_my_test_fliter_set_property (GObject * object, guint prop_id,
const GValue * value, GParamSpec * pspec)
{
GstMyTestFliter *filter = GST_MYTESTFLITER (object);
switch (prop_id) {
case PROP_SILENT:
filter->silent = g_value_get_boolean (value);
break;
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
static void
gst_my_test_fliter_get_property (GObject * object, guint prop_id,
GValue * value, GParamSpec * pspec)
{
GstMyTestFliter *filter = GST_MYTESTFLITER (object);
switch (prop_id) {
case PROP_SILENT:
g_value_set_boolean (value, filter->silent);
break;
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
/* GstElement vmethod implementations */
/* this function handles sink events */
static gboolean
gst_my_test_fliter_sink_event (GstPad * pad, GstObject * parent, GstEvent * event)
{
GstMyTestFliter *filter;
gboolean ret;
filter = GST_MYTESTFLITER (parent);
GST_LOG_OBJECT (filter, "Received %s event: %" GST_PTR_FORMAT,
GST_EVENT_TYPE_NAME (event), event);
switch (GST_EVENT_TYPE (event)) {
case GST_EVENT_CAPS:
{
GstCaps * caps;
gst_event_parse_caps (event, &caps);
/* do something with the caps */
/* and forward */
ret = gst_pad_event_default (pad, parent, event);
break;
}
default:
ret = gst_pad_event_default (pad, parent, event);
break;
}
return ret;
}
/* chain function
* this function does the actual processing
*/
static GstFlowReturn
gst_my_test_fliter_chain (GstPad * pad, GstObject * parent, GstBuffer * buf)
{
GstMyTestFliter *filter;
filter = GST_MYTESTFLITER (parent);
if (filter->silent == FALSE)
g_print ("I'm plugged, therefore I'm in.\n");
/* just push out the incoming buffer without touching it */
return gst_pad_push (filter->srcpad, buf);
}
/* entry point to initialize the plug-in
* initialize the plug-in itself
* register the element factories and other features
*/
static gboolean
mytestfliter_init (GstPlugin * mytestfliter)
{
/* debug category for fltering log messages
*
* exchange the string 'Template mytestfliter' with your description
*/
GST_DEBUG_CATEGORY_INIT (gst_my_test_fliter_debug, "mytestfliter",
0, "Template mytestfliter");
return gst_element_register (mytestfliter, "mytestfliter", GST_RANK_NONE,
GST_TYPE_MYTESTFLITER);
}
/* PACKAGE: this is usually set by autotools depending on some _INIT macro
* in configure.ac and then written into and defined in config.h, but we can
* just set it ourselves here in case someone doesn't use autotools to
* compile this code. GST_PLUGIN_DEFINE needs PACKAGE to be defined.
*/
#ifndef PACKAGE
#define PACKAGE "myfirstmytestfliter"
#endif
/* gstreamer looks for this structure to register mytestfliters
*
* exchange the string 'Template mytestfliter' with your mytestfliter description
*/
GST_PLUGIN_DEFINE (
GST_VERSION_MAJOR,
GST_VERSION_MINOR,
mytestfliter,
"Template mytestfliter",
mytestfliter_init,
PACKAGE_VERSION,
GST_LICENSE,
GST_PACKAGE_NAME,
GST_PACKAGE_ORIGIN
)
解下来，编译Plugin生成对应的so文件。首先在gst-template\gst-plugin\meson.build中添加编译配置
mytestfilter_sources = [
'src/gstmytestfliter.c',
]
gstmytestfliter = library('gstmytestfliter',
mytestfilter_sources,
c_args: plugin_c_args,
dependencies : [gst_dep],
install : true,
install_dir : plugins_install_dir,
)
然后在gst-template目录下（就是仓库的目录）编译这个插件，前提需要安装好Gstreamer的开发环境。
meson build
ninja -C build
编译成功 输出如下内容，生成libgstmyfilter.so就是自定义的插件了。
ninja: Entering directory `build'
[4/4] Linking target gst-plugin/libgstmyfilter.so.
通过工具查看自定义插件信息
把libgstmyfilter.so 拷贝到 /usr/lib/x86_64-linux-gnu/gstreamer-1.0 目录下 (gstreamer库的安装路径，根据实际情况去copy）
copy完后，通过gst-inspect-1.0验证一下
gst-inspect-1.0 mytestfliter
# 输出如下信息：
Factory Details:
Rank none (0)
Long-name MyTestFliter
Klass FIXME:Generic
Description FIXME:Generic Template Element
Author <<[email protected]>>
Plugin Details:
Name mytestfliter
Description Template mytestfliter
Filename /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstmytestfliter.so
Version 1.17.0.1
License LGPL
Source module gst-template-plugin
Binary package GStreamer template Plug-ins
Origin URL https://gstreamer.freedesktop.org
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstMyTestFliter
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
ANY
SINK template: 'sink'
Availability: Always
Capabilities:
ANY
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
name : The name of the object
flags: 可读, 可写
String. Default: "mytestfliter0"
parent : The parent of the object
flags: 可读, 可写
Object of type "GstObject"
silent : Produce verbose output ?
flags: 可读, 可写
Boolean. Default: false
调用自定义Plugin
useplugin.c ，在这个程序中调用了mytestfliter这个插件，设置其silent属性为false。这样在插件收到数据时，就会输出"I’m plugged, therefore I’m in"。插件中实现源码如下
/* chain function
* this function does the actual processing
*/
static GstFlowReturn
gst_my_test_fliter_chain (GstPad * pad, GstObject * parent, GstBuffer * buf)
{
GstMyTestFliter *filter;
filter = GST_MYTESTFLITER (parent);
if (filter->silent == FALSE)
g_print ("I'm plugged, therefore I'm in.\n");
/* just push out the incoming buffer without touching it */
return gst_pad_push (filter->srcpad, buf);
}
下面是useplugin.c 的源码中，视频文件的路径替换为自己实际的路径即可。
#include <gst/gst.h>
typedef struct _CustomData
{
GstElement* pipeline;
GstElement* source;
GstElement* convert;
GstElement* resample;
GstElement* sink;
GstElement* mytestfilter;
} CustomData;
static void pad_added_handler(GstElement* src, GstPad* pad, CustomData* data);
int tutorial_main(int argc, char* argv[])
{
CustomData data;
GstBus* bus;
GstMessage* msg;
GstStateChangeReturn ret;
gboolean terminate = FALSE;
// init
gst_init(&argc, &argv);
// source(pad) -> (sink)convert -> resample -> sink
// decode element
data.source = gst_element_factory_make("uridecodebin", "source");
// convet audio fromat
data.convert = gst_element_factory_make("videoconvert", "convert");
// 重采样
data.resample = gst_element_factory_make("audioresample", "resample");
// sink
data.sink = gst_element_factory_make("autovideosink", "sink");
// test plugin
data.mytestfilter = gst_element_factory_make("mytestfliter", "mytest");
// Create pipeline
data.pipeline = gst_pipeline_new("test-pipeline");
// Check Create Success
if (!data.pipeline
|| !data.source
|| !data.convert
|| !data.resample
|| !data.sink) {
g_printerr("Not all elements could created\n");
return -1;
}
// Add Element to pipeline
gst_bin_add_many(GST_BIN(data.pipeline),
data.source, data.convert, data.resample, data.mytestfilter, data.sink, NULL);
if(!gst_element_link_many(data.convert, data.mytestfilter, data.sink, NULL)){
g_printerr("Element could not be linked\n");
gst_object_unref(data.pipeline);
return -1;
}
// 注意，这里将视频文件，替换为本地实际的视频文件
g_object_set(data.source, "uri",
"file:///home/linduo/test.mp4",
NULL);
g_object_set(data.mytestfilter, "silent", FALSE, NULL);
g_signal_connect(data.source, "pad-added",
G_CALLBACK(pad_added_handler), &data);
// Start pipeline
ret = gst_element_set_state(data.pipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
g_printerr("Start pipeline Error=========");
gst_object_unref(data.pipeline);
return -1;
}
// Listen
bus = gst_element_get_bus(data.pipeline);
do {
msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE,
GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
if (msg != NULL) {
GError *err;
gchar *debug_info;
switch(GST_MESSAGE_TYPE(msg))
{
case GST_MESSAGE_ERROR:
terminate = TRUE;
break;
case GST_MESSAGE_EOS:
terminate = TRUE;
break;
case GST_MESSAGE_STATE_CHANGED:
if (GST_MESSAGE_SRC(msg) == GST_OBJECT(data.pipeline)) {
GstState old_state, new_state, pending_state;
gst_message_parse_state_changed(msg, &old_state,&new_state,
&pending_state);
g_print("Pipeline state changed form %s to %s.\n",
gst_element_state_get_name(old_state),
gst_element_state_get_name(new_state));
}
break;
default:
g_printerr("==========\n");
break;
}
gst_message_unref(msg);
}
} while(!terminate);
gst_object_unref(bus);
gst_element_set_state(data.pipeline, GST_STATE_NULL);
gst_object_unref(data.pipeline);
return 0;
}
static void
pad_added_handler(GstElement* src, GstPad* new_pad, CustomData* data)
{
// Get Convert Sink pad
GstPad* sink_pad = gst_element_get_static_pad(data->convert, "sink");
GstPadLinkReturn ret;
GstCaps* new_pad_caps = NULL;
GstStructure* new_pad_strut = NULL;
const gchar* new_pad_type = NULL;
g_print ("Received new pad '%s' from '%s':\n", GST_PAD_NAME (new_pad),
GST_ELEMENT_NAME (src));
if (gst_pad_is_linked(sink_pad)) {
g_print("We are already linked\n");
goto exit;
}
// 获取当前的Caps
new_pad_caps = gst_pad_get_current_caps(new_pad);
// 获取Caps中的结构体
new_pad_strut = gst_caps_get_structure(new_pad_caps, 0);
// 获取Type类型
new_pad_type = gst_structure_get_name(new_pad_strut);
// 如果Type类型中有audio/x-raw（音频数据）
// if (!g_str_has_prefix(new_pad_type , "audio/x-raw")) {
// g_print("Has not type %s", new_pad_type);
// goto exit;
// }
ret = gst_pad_link(new_pad, sink_pad);
if (GST_PAD_LINK_FAILED(ret)) {
g_print("Link Error\n");
} else {
g_print("Link succeeded [%s]\n", new_pad_type);
}
exit:
if (new_pad_caps != NULL)
gst_caps_unref(new_pad_caps);
gst_object_unref(sink_pad);
}
int main(int argc, char* argv[])
{
return tutorial_main(argc, argv);
}
编译测试程序
gcc useplugin.c -o useplugin `pkg-config --cflags --libs gstreamer-1.0`
执行
./useplugin
Pipeline state changed form NULL to READY.
Received new pad 'src_0' from 'source':
Link succeeded [video/x-raw]
Received new pad 'src_1' from 'source':
We are already linked
I'm plugged, therefore I'm in.
Pipeline state changed form READY to PAUSED.
Pipeline state changed form PAUSED to PLAYING.
I'm plugged, therefore I'm in.
I'm plugged, therefore I'm in.
I'm plugged, therefore I'm in.
I'm plugged, therefore I'm in
...