FFmpeg is an open source artifact for streaming media. It is omnipotent for video conversion, clipping, and streaming. Many systems are developed based on it. You can use FFplay for streaming, but it is not conducive to integrating into your own code, so Vlc.DotNet is chosen for streaming.
In use, only using the default parameters will lead to high overall overhead, occupy a lot of network resources, hardware resources, and cause long delays, so the push-pull flow parameters must be optimized.
1. Streaming parameters
-f dshow -i video="USB Camera":audio= "Internal MIC" -vcodec libx264 -preset:v ultrafast -tune:v zerolatency -crf 28 -acodec aac -s 1280x720 -g 1 -keyint_min 2 -f rtsp -rtsp_transport tcp rtsp://192.168.0.66:554/video
-vcodec libx264 -preset:v ultrafast -tune:v zerolatency -crf 28 This section is video encoding, these parameters use CPU encoding, the overhead is small, and the image quality is generally tolerable. If you can use the GPU, the effect will be better
-s 1280x720 is the resolution setting, if the overall system performance is high, it can be set to the highest resolution of the camera.
-g 1 -keyint_min 2 Interval frame setting, which is beneficial to realize "second open" and reduce connection time.
-rtsp_transport tcp and rtsp transmission uses TCP mode, which is conducive to transmission reliability and reduces system overhead.
The colon ":" in the middle of the audio and video device tells ffmpeg to encode audio and video together to ensure audio and video synchronization and reduce system overhead.
2. Streaming parameters
String[] option = {
":audio-desync=0,
":network-caching=100",
":rtsp-tcp",
":clock-synchro=0",
":live-caching=0",
":file-caching=0",
":grayscale"
};
vlcObj.Play(new Uri("rtsp://192.168.0.66:554/video"), option);audio-desync: The audio is not delayed, the larger the value, the more wrong the audio will be with the video
network-caching: network caching, the larger the value, the smoother it is, but the delay increases
:rtsp-tcp: consistent with ffmpeg streaming configuration
clock-synchro: clock synchronization 0 (disabled), 1 (enabled)
live-caching: live caching
file-caching: file caching
grayscale: video grayscale
3. Final effect
Through the above optimization, the delay in the rtsp stand-alone playback test has been optimized from less than 3 seconds to less than 1 second, and the effect is remarkable. With audio and video synchronization, the "mouth shape" is accurate, and there is no need to adjust the audio delay.