Summary
In the previous article , we described how multiple data transmission element connected to construct a pipline, carried out. So GStreamer is a normal element of data transfer between the means by which to ensure that? Today will introduce how to use GStreamer is Pad to control the transmission of data.
Pad
We know, pad is the interface between the data element, a src pad can only be connected to a sink pad. Each data element may be filtered through the pad, receiving the supported data types. Pad support described by Pad The Capabilities (Pad Caps for short) data type. E.g:
- Represents a resolution of 300x200, frame rate of 30fps Caps RGB video:
“video/x-raw,format=RGB,width=300,height=200,framerate=30/1”
- It represents a sampling bit width is 16 bits, the sampling rate of 44.1kHz, two-channel PCM audio Caps:
“audio/x-raw,format=S16LE,rate=44100,channels=2”
- Or directly coded data format described Voribis, VP8:
“audio/x-vorbis” "video/x-vp8"
Pad can support a plurality of types of Caps (such as a video sink may support data RGB or YUV format, simultaneously), and can specify a range Caps supported data (such as audio sink supports a sampling rate of 1 ~ 48k). However, in a Pipeline, the data transmitted between the Pad type must be unique. GStreamer element when making the connection, choose a type supported by both parties through consultation (negotiation) way.
Therefore, in order to be able to make the two Element proper connection, there must be an intersection between the two sides Pad Caps to select the same data type in the negotiation phase, which is the main role of Pad Caps. In actual use, we can see the Element supported Pad Caps by gst-inspect tools, so we know what to do when the connection error.
Pad Templates (template)
We have used gst_element_factory_make () interface to create Element, the internal interface will create a factory Element, and then create an Element by factory methods. As most of the Element will need to create a similar Pad, so GStreame defined Pad Template, Pad Template Element is included in the factory, when creating Element, for fast creation Pad.
Pad Template contains all the Caps a Pad can support. By Pad Template, we can quickly determine whether two pad can be connected (for example, two elements provide only a sink template, between this element is not connected, so there is no need to further determine Pad Caps).
Because Pad Template belong Element factory, so we can directly use gst-inspect view its properties, but the actual Pad Element will be instantiated, concrete Pad Caps will only be determined after consultation depending on the state of the Element is located.
Pad Templates Capabilities examples
We look at an example "gst-inspect-1.0 alsasink" of (different platforms will be different):
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
audio/x-raw
format: S16LE
layout: interleaved
rate: [ 1, 48000 ]
channels: [ 1, 2 ]
audio/x-ac3
framed: true
alsasink provides only a sink template, you can create a sink pad, and has always been there. It supports two types of audio data: 16-bit PCM (audio / x-raw), the sampling rate 1 ~ 48k, 1-2 channel and AC3 (audio / x-ac3) frame data.
Look a "gst-inspect-1.0 videotestsrc" Examples:
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, NV16, NV24, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16, BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, I420_10LE, I420_10BE, I422_10LE, I422_10BE, Y444_10LE, Y444_10BE, GBR, GBR_10LE, GBR_10BE }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-bayer
format: { bggr, rggb, grbg, gbrg }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
videotestsrc only provides a data type src template for creating src pad, pad supports multiple formats, you can specify the output parameter or Caps Filter specified.
Pad Availability (effectiveness)
Pad Template shown in the example above are always present (Availability: Always), Pad has been created is valid. But some Element dynamically add or delete Pad input data and subsequent Element, therefore GStreamer provides the status of three kinds of Pad validity: Always, Sometimes, On request.
Always Pad
After being initialized in the presence element pad, it referred to always pad or static pad.
Sometimes Pad
Depending on the input pad data generated, it is referred Sometimes pad, common in various file formats parser. E.g. for parsing the mp4 file qtdemux: "gst-inspect-1.0 qtdemux"
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/quicktime
video/mj2
audio/x-m4a
application/x-3gp
SRC template: 'video_%u'
Availability: Sometimes
Capabilities:
ANY
SRC template: 'audio_%u'
Availability: Sometimes
Capabilities:
ANY
SRC template: 'subtitle_%u'
Availability: Sometimes
Capabilities:
ANY
Only when we read the data from the mp4 file, we can know how many audio, video, subtitles included in this document, these src pad are sometimes pad.
Request Pad
pad created on demand is called request pad, common in a merger or generate multiplexed data. For example, for conversion to N tee 1: "gst-inspect-1.0 tee"
Pad Templates:
...
SRC template: 'src_%u'
Availability: On request
Has request_new_pad() function: gst_tee_request_new_pad
Capabilities:
ANY
When we need to use the same video streams simultaneously display and storage, this time we need to use the tee, tee element in the creation of, we do not know how many src pad pipeline needs, you need to request a subsequent element src pad.
Sample Code
GStreamer provides gst-inspect tools to view Pad Templates element provided, but you can not see the element in a different state of Pad supported data types, the following code, we can see the changes Pad Caps in different states.
#include <gst/gst.h> /* Functions below print the Capabilities in a human-friendly format */ static gboolean print_field (GQuark field, const GValue * value, gpointer pfx) { gchar *str = gst_value_serialize (value); g_print ("%s %15s: %s\n", (gchar *) pfx, g_quark_to_string (field), str); g_free (str); return TRUE; } static void print_caps (const GstCaps * caps, const gchar * pfx) { guint i; g_return_if_fail (caps != NULL); if (gst_caps_is_any (caps)) { g_print ("%sANY\n", pfx); return; } if (gst_caps_is_empty (caps)) { g_print ("%sEMPTY\n", pfx); return; } for (i = 0; i < gst_caps_get_size (caps); i++) { GstStructure *structure = gst_caps_get_structure (caps, i); g_print ("%s%s\n", pfx, gst_structure_get_name (structure)); gst_structure_foreach (structure, print_field, (gpointer) pfx); } } /* Prints information about a Pad Template, including its Capabilities */ static void print_pad_templates_information (GstElementFactory * factory) { const GList *pads; GstStaticPadTemplate *padtemplate; g_print ("Pad Templates for %s:\n", gst_element_factory_get_longname (factory)); if (!gst_element_factory_get_num_pad_templates (factory)) { g_print (" none\n"); return; } pads = gst_element_factory_get_static_pad_templates (factory); while (pads) { padtemplate = pads->data; pads = g_list_next (pads); if (padtemplate->direction == GST_PAD_SRC) g_print (" SRC template: '%s'\n", padtemplate->name_template); else if (padtemplate->direction == GST_PAD_SINK) g_print (" SINK template: '%s'\n", padtemplate->name_template); else g_print (" UNKNOWN!!! template: '%s'\n", padtemplate->name_template); if (padtemplate->presence == GST_PAD_ALWAYS) g_print (" Availability: Always\n"); else if (padtemplate->presence == GST_PAD_SOMETIMES) g_print (" Availability: Sometimes\n"); else if (padtemplate->presence == GST_PAD_REQUEST) g_print (" Availability: On request\n"); else g_print (" Availability: UNKNOWN!!!\n"); if (padtemplate->static_caps.string) { GstCaps *caps; g_print (" Capabilities:\n"); caps = gst_static_caps_get (&padtemplate->static_caps); print_caps (caps, " "); gst_caps_unref (caps); } g_print ("\n"); } } /* Shows the CURRENT capabilities of the requested pad in the given element */ static void print_pad_capabilities (GstElement *element, gchar *pad_name) { GstPad *pad = NULL; GstCaps *caps = NULL; /* Retrieve pad */ pad = gst_element_get_static_pad (element, pad_name); if (!pad) { g_printerr ("Could not retrieve pad '%s'\n", pad_name); return; } /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */ caps = gst_pad_get_current_caps (pad); if (!caps) caps = gst_pad_query_caps (pad, NULL); /* Print and free */ g_print ("Caps for the %s pad:\n", pad_name); print_caps (caps, " "); gst_caps_unref (caps); gst_object_unref (pad); } int main(int argc, char *argv[]) { GstElement *pipeline, *source, *sink; GstElementFactory *source_factory, *sink_factory; GstBus *bus; GstMessage *msg; GstStateChangeReturn ret; gboolean terminate = FALSE; /* Initialize GStreamer */ gst_init (&argc, &argv); /* Create the element factories */ source_factory = gst_element_factory_find ("audiotestsrc"); sink_factory = gst_element_factory_find ("autoaudiosink"); if (!source_factory || !sink_factory) { g_printerr ("Not all element factories could be created.\n"); return -1; } /* Print information about the pad templates of these factories */ print_pad_templates_information (source_factory); print_pad_templates_information (sink_factory); /* Ask the factories to instantiate actual elements */ source = gst_element_factory_create (source_factory, "source"); sink = gst_element_factory_create (sink_factory, "sink"); /* Create the empty pipeline */ pipeline = gst_pipeline_new ("test-pipeline"); if (!pipeline || !source || !sink) { g_printerr ("Not all elements could be created.\n"); return -1; } /* Build the pipeline */ gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL); if (gst_element_link (source, sink) != TRUE) { g_printerr ("Elements could not be linked.\n"); gst_object_unref (pipeline); return -1; } /* Print initial negotiated caps (in NULL state) */ g_print ("In NULL state:\n"); print_pad_capabilities (sink, "sink"); /* Start playing */ ret = gst_element_set_state (pipeline, GST_STATE_PLAYING); if (ret == GST_STATE_CHANGE_FAILURE) { g_printerr ("Unable to set the pipeline to the playing state (check the bus for error messages).\n"); } /* Wait until error, EOS or State Change */ bus = gst_element_get_bus (pipeline); do { msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS | GST_MESSAGE_STATE_CHANGED); /* Parse message */ if (msg != NULL) { GError *err; gchar *debug_info; switch (GST_MESSAGE_TYPE (msg)) { case GST_MESSAGE_ERROR: gst_message_parse_error (msg, &err, &debug_info); g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message); g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none"); g_clear_error (&err); g_free (debug_info); terminate = TRUE; break; case GST_MESSAGE_EOS: g_print ("End-Of-Stream reached.\n"); terminate = TRUE; break; case GST_MESSAGE_STATE_CHANGED: /* We are only interested in state-changed messages from the pipeline */ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) { GstState old_state, new_state, pending_state; gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state); g_print ("\nPipeline state changed from %s to %s:\n", gst_element_state_get_name (old_state), gst_element_state_get_name (new_state)); /* Print the current capabilities of the sink element */ print_pad_capabilities (sink, "sink"); } break; default: /* We should not reach here because we only asked for ERRORs, EOS and STATE_CHANGED */ g_printerr ("Unexpected message received.\n"); break; } gst_message_unref (msg); } } while (!terminate); /* Free resources */ gst_object_unref (bus); gst_element_set_state (pipeline, GST_STATE_NULL); gst_object_unref (pipeline); gst_object_unref (source_factory); gst_object_unref (sink_factory); return 0; }
Save the source code for basic-tutorial-3.c, execute the following command to compile the results obtained:
$ gcc basic-tutorial-3.c -o basic-tutorial-3 `pkg-config --cflags --libs gstreamer-1.0`
Source code analysis
Readable information output
print_field, print_caps and print_pad_templates_information achieve similar functionality, the print data structure GStreamer, you can view the GStreamer GstCaps interfaces for more information.
/* Shows the CURRENT capabilities of the requested pad in the given element */ static void print_pad_capabilities (GstElement *element, gchar *pad_name) { GstPad *pad = NULL; GstCaps *caps = NULL; /* Retrieve pad */ pad = gst_element_get_static_pad (element, pad_name); if (!pad) { g_printerr ("Could not retrieve pad '%s'\n", pad_name); return; } /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */ caps = gst_pad_get_current_caps (pad); if (!caps) caps = gst_pad_query_caps (pad, NULL); /* Print and free */ g_print ("Caps for the %s pad:\n", pad_name); print_caps (caps, " "); gst_caps_unref (caps); gst_object_unref (pad); }
Because the source and sink we use have static (always) pad, so here use gst_element_get_static_pad () Gets Pad, otherwise you can use gst_element_foreach_pad () or gst_element_iterate_pads () Gets Pad dynamically created.
Then use gst_pad_get_current_caps () Gets the current pad caps, have different results depending on the state of element, even caps may not exist. If not, we can get caps currently supported by gst_pad_query_caps (), when the element is in the NULL state, the caps for the Pad Template supported caps, its value may vary depending on the state of change.
Gets Element Factory
/* Create the element factories */ source_factory = gst_element_factory_find ("audiotestsrc"); sink_factory = gst_element_factory_find ("autoaudiosink"); if (!source_factory || !sink_factory) { g_printerr ("Not all element factories could be created.\n"); return -1; } /* Print information about the pad templates of these factories */ print_pad_templates_information (source_factory); print_pad_templates_information (sink_factory); /* Ask the factories to instantiate actual elements */ source = gst_element_factory_create (source_factory, "source"); sink = gst_element_factory_create (sink_factory, "sink");
When using gst_element_factory_make () to create an interface element, the application need not concern element factory. Here, since the Pad Template Data Element project, so we first factories were created corresponding factory instance (GstElementFactory), then by acquiring Pad Template and create element.
As used herein gst_element_factory_find () to find "audiotestsrc" factory, then by gst_element_factory_create () to create a source element. Previously used gst_element_factory_make () is gst_element_factory_find () + gst_element_factory_create () is a simplified version.
State-Changed message processing
Pipeline creation same as the other examples, the process of this new cases of state change.
case GST_MESSAGE_STATE_CHANGED: /* We are only interested in state-changed messages from the pipeline */ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) { GstState old_state, new_state, pending_state; gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state); g_print ("\nPipeline state changed from %s to %s:\n", gst_element_state_get_name (old_state), gst_element_state_get_name (new_state)); /* Print the current capabilities of the sink element */ print_pad_capabilities (sink, "sink"); } break;
Since we joined the GST_MESSAGE_STATE_CHANGED in gst_bus_timed_pop_filtered () in, so we will receive the message state changes. When a change in state, the information output pad caps sink element in the current state.
Output analysis
Pad Templates for Audio test source:
SRC template: 'src'
Availability: Always
Capabilities:
audio/x-raw
format: { S16LE, S32LE, F32LE, F64LE }
layout: interleaved
rate: [ 1, 2147483647 ]
channels: [ 1, 2 ]
Pad Templates for Auto audio sink:
SINK template: 'sink'
Availability: Always
Capabilities:
ANY
The first is "audiotestsrc" and "autoaudiosink" pad templates of information, the output gst-inspect the same.
In NULL state:
Caps for the sink pad:
ANY
NULL state initialization state Element at this time, "autoaudiosink" The sink pad caps with the same Pad Template, supports all formats.
Pipeline state changed from NULL to READY:
Caps for the sink pad:
audio/x-raw
format: { S16LE, S16BE, F32LE, F32BE, S32LE, S32BE, S24LE, S24BE, S24_32LE, S24_32BE, U8 }
layout: interleaved
rate: [ 1, 2147483647 ]
channels: [ 1, 32 ]
audio/x-alaw
rate: [ 1, 2147483647 ]
channels: [ 1, 32 ]
audio/x-mulaw
rate: [ 1, 2147483647 ]
channels: [ 1, 32 ]
NULL state to READY from, GStreamer will get all types of audio output devices supported, where you can see the sink pad caps lists the types of output devices can support.
Pipeline state changed from READY to PAUSED:
Caps for the sink pad:
audio/x-raw
format: S16LE
layout: interleaved
rate: 44100
channels: 1
Pipeline state changed from PAUSED to PLAYING:
Caps for the sink pad:
audio/x-raw
format: S16LE
layout: interleaved
rate: 44100
channels: 1
READY state to PAUSED from, GStreamer will negotiate a all element types are supported. When entering state PLAYING, sink type will use the negotiated data transfer.
to sum up
In this tutorial, we mastered:
- What is the Pad Capabilities and Pad Template Capabilities.
- Pad effectiveness category.
- How to get the current caps by gst_pad_get_current_caps () and gst_pad_query_caps ().
- Pad Capabilities change in different states of the element.
- How to use gst-inspect tools View of Pad Caps element.
Quote
https://gstreamer.freedesktop.org/documentation/tutorials/basic/media-formats-and-pad-capabilities.html?gi-language=c
https://gstreamer.freedesktop.org/documentation/tutorials/basic/multithreading-and-pad-availability.html
https://gstreamer.freedesktop.org/documentation/gstreamer/gstcaps.html?gi-language=c