Android SeekBar controls video playback progress (2) - seekTo() is not accurate

Introduction

In the previous article, we introduced the use of SeekBar to control video playback . During use, we found that for some videos, SeekBarwhen we drag the progress bar to adjust the playback progress, after adjusting to the specified position, the progress bar will jump back, and will not Play continues where we dragged.

Internet search learned that VideoView.seekTo()the strategy of the method is determined. Look at seekTo()the method in detail:

seekTo()

1. The following is VideoView.seekTo(int msec)the code implementation, and we realize the progress adjustment by calling this method. By looking at the code, we know that the method actually calls the MediaPlayer.seekTo(msec);

@Override
public void seekTo(int msec) {
    if (isInPlaybackState()) {
        mMediaPlayer.seekTo(msec);
        mSeekWhenPrepared = 0;
    } else {
        mSeekWhenPrepared = msec;
    }
}

2. Go ahead and look at MediaPlayer.seekTo(msec);the implementation of the method that calls the method, which is seekTo(long msec, @SeekMode int mode)the default .modeSEEK_PREVIOUS_SYNC

/**
 * Seeks to specified time position.
 * Same as {@link #seekTo(long, int)} with {@code mode = SEEK_PREVIOUS_SYNC}.
 *
 * @param msec the offset in milliseconds from the start to seek to
 * @throws IllegalStateException if the internal player engine has not been
 * initialized
 */
public void seekTo(int msec) throws IllegalStateException {
    
    
    seekTo(msec, SEEK_PREVIOUS_SYNC /* mode */);
}

/**
 * Moves the media to specified time position by considering the given mode.
 * <p>
 * When seekTo is finished, the user will be notified via OnSeekComplete supplied by the user.
 * There is at most one active seekTo processed at any time. If there is a to-be-completed
 * seekTo, new seekTo requests will be queued in such a way that only the last request
 * is kept. When current seekTo is completed, the queued request will be processed if
 * that request is different from just-finished seekTo operation, i.e., the requested
 * position or mode is different.
 *
 * @param msec the offset in milliseconds from the start to seek to.
 * When seeking to the given time position, there is no guarantee that the data source
 * has a frame located at the position. When this happens, a frame nearby will be rendered.
 * If msec is negative, time position zero will be used.
 * If msec is larger than duration, duration will be used.
 * @param mode the mode indicating where exactly to seek to.
 * Use {@link #SEEK_PREVIOUS_SYNC} if one wants to seek to a sync frame
 * that has a timestamp earlier than or the same as msec. Use
 * {@link #SEEK_NEXT_SYNC} if one wants to seek to a sync frame
 * that has a timestamp later than or the same as msec. Use
 * {@link #SEEK_CLOSEST_SYNC} if one wants to seek to a sync frame
 * that has a timestamp closest to or the same as msec. Use
 * {@link #SEEK_CLOSEST} if one wants to seek to a frame that may
 * or may not be a sync frame but is closest to or the same as msec.
 * {@link #SEEK_CLOSEST} often has larger performance overhead compared
 * to the other options if there is no sync frame located at msec.
 * @throws IllegalStateException if the internal player engine has not been
 * initialized
 * @throws IllegalArgumentException if the mode is invalid.
 */
public void seekTo(long msec, @SeekMode int mode) {
    
    
    if (mode < SEEK_PREVIOUS_SYNC || mode > SEEK_CLOSEST) {
    
    
        final String msg = "Illegal seek mode: " + mode;
        throw new IllegalArgumentException(msg);
    }
    // TODO: pass long to native, instead of truncating here.
    if (msec > Integer.MAX_VALUE) {
    
    
        Log.w(TAG, "seekTo offset " + msec + " is too large, cap to " + Integer.MAX_VALUE);
        msec = Integer.MAX_VALUE;
    } else if (msec < Integer.MIN_VALUE) {
    
    
        Log.w(TAG, "seekTo offset " + msec + " is too small, cap to " + Integer.MIN_VALUE);
        msec = Integer.MIN_VALUE;
    }
    _seekTo(msec, mode);
}

3. SeekModeThere are following modes,

SEEK_PREVIOUS_SYNC: seek to the previous keyframe
SEEK_NEXT_SYNC: seek to the next keyframe
SEEK_CLOSEST_SYNC: seek to the nearest keyframe
SEEK_CLOSEST: seek to the nearest frame (does not need to be a keyframe)

    /**
     * Seek modes used in method seekTo(long, int) to move media position
     * to a specified location.
     *
     * Do not change these mode values without updating their counterparts
     * in include/media/IMediaSource.h!
     */
    /**
     * This mode is used with {@link #seekTo(long, int)} to move media position to
     * a sync (or key) frame associated with a data source that is located
     * right before or at the given time.
     *
     * @see #seekTo(long, int)
     */
    public static final int SEEK_PREVIOUS_SYNC    = 0x00;
    /**
     * This mode is used with {@link #seekTo(long, int)} to move media position to
     * a sync (or key) frame associated with a data source that is located
     * right after or at the given time.
     *
     * @see #seekTo(long, int)
     */
    public static final int SEEK_NEXT_SYNC        = 0x01;
    /**
     * This mode is used with {@link #seekTo(long, int)} to move media position to
     * a sync (or key) frame associated with a data source that is located
     * closest to (in time) or at the given time.
     *
     * @see #seekTo(long, int)
     */
    public static final int SEEK_CLOSEST_SYNC     = 0x02;
    /**
     * This mode is used with {@link #seekTo(long, int)} to move media position to
     * a frame (not necessarily a key frame) associated with a data source that
     * is located closest to or at the given time.
     *
     * @see #seekTo(long, int)
     */
    public static final int SEEK_CLOSEST          = 0x03;

4. Therefore, when the video jumps to the corresponding position and lacks a key frame, calling the seekTo method cannot start playing at the current position. At this time, it will find the key frame position closest to the specified position and start playing.
   What we call through the seekTo function is actually the default mode-- SEEK_PREVIOUS_SYNC , at this time we will search for the previous keyframe of the position. Therefore, after adjusting the progress of the video, the video will jump back for a while, and will not continue to play at the position we dragged.

Video Frames and Video Keyframes

The above method mentioned frame and key frame, let's briefly introduce the relationship and difference between the two. We know that a video is composed of images frame by frame, and key frames are some of the frames. Ideally, we would turn all normal frames into keyframes, so there would be no backlash when adjusting the progress of the video.

Solution

method one

According to SeekModethe description of several modes, it is specified when modecalling SEEK_CLOSEST.

Method Two

Process the video source file to increase the number of key frames. Use FFmpeg to process the video and add key frames of the video.

1. First, we check the number of key frames in the current video with the following command:

ffprobe -show_frames /Users/Admin/Desktop/test.mp4 >video_log.txt

Output video information to a text file, open video_log.txtthe file, search for keywords pict_type=Ito view key frames. You can see that our current video has only 11 key frames.

2. Add key frames to the video, keyint=30and set a key frame every 30 frames. The command is as follows:

ffmpeg.exe -i "/Users/Admin/Desktop/test.mp4" -c:v libx264 -preset superfast -x264opts keyint=30 -acodec copy -f mp4 "/Users/Admin/Desktop/test_out.mp4"

Use the command in step 1 to view the processed video information. It can be seen that after processing, the number of key frames in our video is 99.