Summary
In the popular media player, you can usually see fast-forward, rewind, slow motion and other functions, this part of the function known as "trick modes (Trick Mode)", these models have in common: both played by modifying rate to achieve the appropriate destination. This article describes how to implement fast-forward, rewind, slow motion and single-frame playback through GStreamer.
GStreamer Seek to Step event
Fast forward (Fast-Forward), Rewind (Fast-Rewind) and slow (Slow-Motion) by modifying the rate of play is to achieve the corresponding object. In GStreamer, the speed as a normal playback speed, the speed will be greater than a 2-fold, 4-fold, 8-fold and so called speed fast forward, slow play is the absolute value of the rate of speed is less than 1, when the playback speed is less than 0:00, proceed upside down.
In GStreamer, we have to control the playback rate and regional Element by step and seek event. Step event allows to skip a specified area and set subsequent playback rate (This rate must be greater than 0). Seek event allows playback to jump to any location in the file, the playback rate may be greater than 0 and less than 0.1 or
the playback time control , we used gst_element_seek_simple to quickly jump to a specified location, this function is the event seek package. Actual use, we first need to construct a seek event, set the absolute starting position and seek the stop position, the stop position can be set to 0, it will seek to perform playback rate until the end. And can seek support by way of the buffer, as well as set a different flag specifies the behavior of seek.
Step event compared to Seek event requires fewer parameters, and easier to modify the playback speed, but not flexible enough. Step event will only act on the final sink, Seek Pipeline event you can act on all of the Element. Step efficiency operation than Seek.
In GStreamer, the single-frame playback (Frame Stepping) with the same fast-forward, but also to achieve through the event. Usually in single-frame play pause state, and transmits configuration each play a step event.
It should be noted, seek event needs to be working sink element (eg: audio sink or video sink), if the direct effect will seek event in Pipeline, Pipeline will automatically be forwarded to all the events sink, if there are multiple sink, it It will cause many times seek. Usually by obtaining video-sink or audio-sink Pipeline, and then sent to the specified seek Event sink, the seek operation is completed. Seek and send a sample configuration time is as follows:
GstEvent *event; gboolean result; ... // construct a seek event to play the media from second 2 to 5, flush // the pipeline to decrease latency. event = gst_event_new_seek (1.0, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH, GST_SEEK_TYPE_SET, 2 * GST_SECOND, GST_SEEK_TYPE_SET, 5 * GST_SECOND); ... result = gst_element_send_event (video_sink, event); if (!result) g_warning ("seek failed"); ...
Sample Code
By following a complete example, to see how GStreamer by step and seek to achieve the appropriate playback speed.
#include <string.h> #include <stdio.h> #include <gst/gst.h> typedef struct _CustomData { GstElement *pipeline; GstElement *video_sink; GMainLoop *loop; gboolean playing; /* Playing or Paused */ gdouble rate; /* Current playback rate (can be negative) */ } CustomData; /* Send seek event to change rate */ static void send_seek_event (CustomData * data) { gint64 position; GstEvent *seek_event; /* Obtain the current position, needed for the seek event */ if (!gst_element_query_position (data->pipeline, GST_FORMAT_TIME, &position)) { g_printerr ("Unable to retrieve current position.\n"); return; } /* Create the seek event */ if (data->rate > 0) { seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE, GST_SEEK_TYPE_SET, position, GST_SEEK_TYPE_END, 0); } else { seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE, GST_SEEK_TYPE_SET, 0, GST_SEEK_TYPE_SET, position); } if (data->video_sink == NULL) { /* If we have not done so, obtain the sink through which we will send the seek events */ g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL); } /* Send the event */ gst_element_send_event (data->video_sink, seek_event); g_print ("Current rate: %g\n", data->rate); } /* Process keyboard input */ static gboolean handle_keyboard (GIOChannel * source, GIOCondition cond, CustomData * data) { gchar *str = NULL; if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) != G_IO_STATUS_NORMAL) { return TRUE; } switch (g_ascii_tolower (str[0])) { case 'p': data->playing = !data->playing; gst_element_set_state (data->pipeline, data->playing ? GST_STATE_PLAYING : GST_STATE_PAUSED); g_print ("Setting state to %s\n", data->playing ? "PLAYING" : "PAUSE"); break; case 's': if (g_ascii_isupper (str[0])) { data->rate *= 2.0; } else { data->rate /= 2.0; } send_seek_event (data); break; case 'd': data->rate *= -1.0; send_seek_event (data); break; case 'n': if (data->video_sink == NULL) { /* If we have not done so, obtain the sink through which we will send the step events */ g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL); } gst_element_send_event (data->video_sink, gst_event_new_step (GST_FORMAT_BUFFERS, 1, ABS (data->rate), TRUE, FALSE)); g_print ("Stepping one frame\n"); break; case 'q': g_main_loop_quit (data->loop); break; default: break; } g_free (str); return TRUE; } int main (int argc, char *argv[]) { CustomData data; GstStateChangeReturn ret; GIOChannel *io_stdin; /* Initialize GStreamer */ gst_init (&argc, &argv); /* Initialize our data structure */ memset (&data, 0, sizeof (data)); /* Print usage map */ g_print ("USAGE: Choose one of the following options, then press enter:\n" " 'P' to toggle between PAUSE and PLAY\n" " 'S' to increase playback speed, 's' to decrease playback speed\n" " 'D' to toggle playback direction\n" " 'N' to move to next frame (in the current direction, better in PAUSE)\n" " 'Q' to quit\n"); /* Build the pipeline */ data.pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL); /* Add a keyboard watch so we get notified of keystrokes */ #ifdef G_OS_WIN32 io_stdin = g_io_channel_win32_new_fd (fileno (stdin)); #else io_stdin = g_io_channel_unix_new (fileno (stdin)); #endif g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc) handle_keyboard, &data); /* Start playing */ ret = gst_element_set_state (data.pipeline, GST_STATE_PLAYING); if (ret == GST_STATE_CHANGE_FAILURE) { g_printerr ("Unable to set the pipeline to the playing state.\n"); gst_object_unref (data.pipeline); return -1; } data.playing = TRUE; data.rate = 1.0; /* Create a GLib Main Loop and set it to run */ data.loop = g_main_loop_new (NULL, FALSE); g_main_loop_run (data.loop); /* Free resources */ g_main_loop_unref (data.loop); g_io_channel_unref (io_stdin); gst_element_set_state (data.pipeline, GST_STATE_NULL); if (data.video_sink != NULL) gst_object_unref (data.video_sink); gst_object_unref (data.pipeline); return 0; }
By following command to compile the executable file can be obtained, the input terminal corresponding instructions in the playback rate can be modified.
gcc basic-tutorial-7.c -o basic-tutorial-7 `pkg-config --cflags --libs gstreamer-1.0`
Source code analysis
In the present embodiment, to create the same as the other examples Pipeline, playbin by playing the file, using the GLib I / O interface to process the keyboard input.
/* Process keyboard input */ static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) { gchar *str = NULL; if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) != G_IO_STATUS_NORMAL) { return TRUE; } switch (g_ascii_tolower (str[0])) { case 'p': data->playing = !data->playing; gst_element_set_state (data->pipeline, data->playing ? GST_STATE_PLAYING : GST_STATE_PAUSED); g_print ("Setting state to %s\n", data->playing ? "PLAYING" : "PAUSE"); break;
When the input terminal P, using gst_element_set_state () Set playback.
case 's': if (g_ascii_isupper (str[0])) { data->rate *= 2.0; } else { data->rate /= 2.0; } send_seek_event (data); break; case 'd': data->rate *= -1.0; send_seek_event (data); break;
And S is increased by reducing the playback speed and s, d for changing the playback direction (rewind), here modified Rate, send_seek_event true calling process.
/* Send seek event to change rate */ static void send_seek_event (CustomData *data) { gint64 position; GstEvent *seek_event; /* Obtain the current position, needed for the seek event */ if (!gst_element_query_position (data->pipeline, GST_FORMAT_TIME, &position)) { g_printerr ("Unable to retrieve current position.\n"); return; }
This function is configured to send a SeekEvent Pipeline to adjust the rate. Because Seek Event will jump to a specified location, but in this case we want to change the rate summary, do not jump to another location, so first get the current playback position by gst_element_query_position ().
/* Create the seek event */ if (data->rate > 0) { seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE, GST_SEEK_TYPE_SET, position, GST_SEEK_TYPE_END, 0); } else { seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE, GST_SEEK_TYPE_SET, 0, GST_SEEK_TYPE_SET, position); }
() Created by gst_event_new_seek SeekEvent, setting a new rate, flag, start position, end position. Note that, the start position requires less than the end position.
if (data->video_sink == NULL) { /* If we have not done so, obtain the sink through which we will send the seek events */ g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL); } /* Send the event */ gst_element_send_event (data->video_sink, seek_event);
As mentioned above, in order to avoid Pipeline perform seek many times, we get video-sink in here, and send it SeekEvent. When we get there until Seek video-sink it is because the actual sink is likely to create only when PLAYING state according to different media types.
Some of these items is the rate changes, the situation with regard to single-frame playback, implementation easier:
case 'n': if (data->video_sink == NULL) { /* If we have not done so, obtain the sink through which we will send the step events */ g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL); } gst_element_send_event (data->video_sink, gst_event_new_step (GST_FORMAT_BUFFERS, 1, ABS (data->rate), TRUE, FALSE)); g_print ("Stepping one frame\n"); break;
We gst_event_new_step () creates StepEvent, and assign only one jump, and do not change the current rate. Single-frame play are usually pause, and then single-frame playback.
These are achieved through the playback rate control GStreamer, in practice, some of the upside Element support is not very good, it can not achieve the desired results.
to sum up
In this article we have mastered:
- How gst_event_new_seek () configured SeekEvent, transmitted to the sink rate of change by gst_element_send_event ().
- How to achieve single-frame play by gst_event_new_step ().
Quote
https://gstreamer.freedesktop.org/documentation/tutorials/basic/playback-speed.html?gi-language=c
https://gstreamer.freedesktop.org/documentation/additional/design/seeking.html?gi-language=c
https://gstreamer.freedesktop.org/documentation/additional/design/framestep.html?gi-language=c