WebRTC brings capabilities such as real-time streaming media and P2P into the front-end of the Web. Developers only need to write simple JavaScript programs to develop rich real-time multimedia applications. This conference wants to share with you some of our practical experience in WebRTC, and how to use WebRTC data channels for P2P streaming media. Finally, we introduce how we design a P2P streaming algorithm with low latency and high bandwidth utilization.
The birth background of WebRTC
We know that real-time video communication is very common now, based on video call tools such as FaceTime and Skype, users can easily conduct video conversations with others. In order to optimize the user experience to the extreme, developers use a large number of technical means to ensure video quality, such as reducing packet loss, recovering from network disconnection, and responding to user network changes in real time. Real-time audio and video communication has relatively high technical requirements for developers, and patent holding companies charge licensing fees to developers and build huge technical barriers. On the other hand, for users, it is necessary to additionally install corresponding plug-ins or application programs, which reduces the user experience, and there is also a risk of being bundled with rogue software. At this time, a technology called WebRTC came into being.
Before talking about WebRTC, let's review the evolution history of Web communication. Before the advent of AJAX, that is, before 2005, if the content needs to be updated, the entire web page must be reloaded. After the emergence of AJAX, by exchanging a small amount of data with the server in the background, AJAX can make the webpage update asynchronously. However, AJAX cannot perform duplex communication with the server, so the server cannot actively push messages to the browser, and can only poll through the browser. The emergence of Websocket has changed this situation, and the browser and server can perform full-duplex communication. Whether it is AJAX or Websocket, data needs to be sent to the server. In order to transmit data between two users, developers need to purchase a server network, which is very costly. A new technology backed by Google - WebRTC has completely changed this situation. WebRTC is the abbreviation of Web Real-Time Communication, which realizes direct real-time communication between browsers without server transfer. Google is committed to making it one of the HTML5 standards, and most browsers currently support it.
Combination of WebRTC and P2P
In 2012, Google's chrome browser officially supported WebRTC natively. Web developers only need a few lines of javascript code to develop rich real-time multimedia applications, and users do not need to install plug-ins, and can chat with each other in real time by directly opening the browser. At this time, some developers with a keen sense of smell began to use WebRTC's data channel technology to do P2P streaming media. For example, a foreign company called peer5. Alan, the founder of our company, also devoted himself to this research when he was working at Tencent, but was disappointed to find that there are still some problems that are difficult to solve when using WebRTC for P2P streaming media. For example, the online time of users is not stable. When users close the page, The data channel of WebRTC is also closed. Then in 2013 and 2014, Firefox and Opera also announced support for WebRTC. At this time, Alan put forward a bold idea. Since the browser is unstable, how about implementing the same protocol in routers and NAS? We all know that routers are turned on 24 hours a day, but most of the time they are idle. If these computing power and network bandwidth can be utilized, it is equivalent to thousands of households being nodes, and your neighbors and even yourself may be on the Internet. To provide acceleration for you to watch videos, it's a cool thing to think about! Therefore, we proposed the concept of crowdsourcing CDN and applied for a patent. In 2015, Tencent's X5 browser kernel and WeChat also provided support. In the same year, we officially announced the establishment of Lixiang Computing.
You may have doubts, will WebRTC really become a mainstream technology in the future? Let's speak with facts, just look at the support of major browsers. It can be seen from the figure that most browsers already support WebRTC, including chrome, firefox and opera, and Microsoft's edge browser partially supports WebRTC. It is certain that there will be more and more applications based on WebRTC in the future.
WebRTC media session principle
We assume that there are two browsers A and B who want to establish a WebRTC peer-to-peer connection. A peer-to-peer connection is a direct media connection between two web browsers. If A wants to actively contact B, it needs to send a message to the signaling server via HTTP first. An SDP, SDP can be understood as a computer business card, the full name is Session Description Protocol, Session Description Protocol, used to describe the media characteristics of peer-to-peer connections. So what is a signaling server? It is like a matchmaker, helping two people who don't know each other match. The SDP sent by browser A is called offer, and the signaling server sends it to browser B, which responds with an SDP object called answer, which is also forwarded to A through the signaling server. After exchanging SDP, the two peers start to try ICE hole punching. After the hole punching is successful, they start negotiating keys, and then they can start a secure media or data session.
ICE punching principle
Due to the limited IP resources provided by IPv4 and the promotion of IPv6, most network devices are still in the intranet and need to be connected to the external internet through NAT devices. The full name of NAT is Network Address Translation, network address translation, a router equipped with NAT software is called a NAT router, and it has at least one valid external global IP address. In this way, when all hosts using local addresses communicate with the outside world, their local addresses must be converted into global IP addresses on the NAT router in order to connect to the Internet. When two peers are in different LANs, they need to know each other's public IP and port first. At this time, a test data packet can be sent to the STUN server first, and the latter responds, indicating the IP address detected in the test data packet, and this address will be returned as a potential candidate address. The browser that gets the candidate address sends it to the peer through the signaling server, and the peer performs the same operation, and then the two parties try to connect with all the candidate addresses obtained, and if they fail, they will use TURN The server is used as a transit server, and the TURN server is only adopted when all alternatives are invalid, because the cost is relatively high. In order to speed up the call establishment time, there is a solution called trickle ice. The idea is that the client collects the candidates and sends them to the other party. For example, the local candidate can be initiated directly without obtaining it through the stun, which reduces the time to complete the connectivity detection.
WebRTC data channel
Next, introduce a more important concept - WebRTC data channel. We do P2P streaming media based on WebRTC, which actually uses the data channel capability. So what exactly is a data channel? While much of the hype about WebRTC has focused on its support for real-time audio and video communications, designers have long hoped that it would also support real-time data transmission. Compared with Websocket and HTTP, the data channel supports connections with large traffic and low latency, and has the advantages of stability and reliability. Moreover, the interface of data channel is the same as websocket, which also sends data through send and receives data through ommessage. So how to control the reliability of data channel data transmission? Through the dataChannelOptions javascript object just mentioned, the data channel can be switched between the advantages of UDP or TCP, such as making data transmission more stable and reliable, or faster. There are several important fields: ordered: set whether the data should be received in the order in which they were sent, maxRetransmitTime: set how long to resend when the data fails to be sent, maxRetransmits: set the maximum number of retransmissions when the data fails to be sent. The main thing is to configure ordered. When set to true, the data channel behaves more like TCP, and when it is false, it behaves more like UDP.
WebRTC and P2P streaming
After figuring out the data channel of WebRTC, we can use it for P2P streaming. In this regard, there is already a well-known open source project developed by foreign masters: WebTorrent, which has more than 10,000 stars on github. WebTorrent is an open-source js framework based on WebRTC and BitTorrent protocols. It is written entirely in javascript and can run on Node.js and browsers at the same time. Because it is based on WebRTC, WebTorrent can run on browsers without installing any plug-ins. Also supports Chrome, Firefox and Opera browsers. However, because it is based on the BT protocol, it is a pull-based algorithm. This algorithm is a random capture strategy, which randomly captures the buffers of other nodes, but there is a problem: the captured buffers are not necessarily It is needed at present, and it is not necessarily needed by other nodes, and it will waste downlink bandwidth and uplink bandwidth of other nodes, thus causing the problems of "bandwidth hunger" and "content hunger" at the same time. The following introduces an improved version of the pull-based algorithm - FirstAid algorithm. FirstAid is based on window sliding, which triggers window sliding every once in a while, and each window can be divided into three sections: urgent, normal and prefetch. As the name suggests, urgent is the buffer closest to the playback time, so the priority level is the highest, normal and prefetch Decreasing priority. When the parent node transmits the buffer for the child node, it will first meet the urgent level requirements, and suspend the normal level, so the most urgent needs will be satisfied first. When the urgent needs of the child nodes are met, they need to go back and make up for them competitors' needs in order to achieve a state of mutual benefit. Contrary to the idea of the pull-based algorithm just now is the push-based algorithm, among which the more representative one is the FashMesh algorithm, a P2P algorithm proposed by scholars at the University of Science and Technology of Hong Kong. FashMesh is based on an algorithm called Streaming Mesh, which divides the data flow of the source node into multiple sub-flows, and transmits the mesh source continuously to the child nodes through multiple spanning trees. The advantages of this algorithm are low delay and bandwidth utilization. high. Fast Mesh can also dynamically adjust the network topology according to the uplink bandwidth of each child node, so that nodes with large uplink bandwidth are closer to the source node, so as to make full use of the existing capabilities of the network. According to a comparative experiment, FastMesh may be the best among many P2P algorithms. But this algorithm also has disadvantages. When a node enters or leaves the network, it needs to readjust the topology structure, so it is not suitable for the situation where the nodes change greatly. Mesh can also dynamically adjust the network topology according to the uplink bandwidth of each child node, so that nodes with large uplink bandwidth are closer to the source node, so as to make full use of the existing capabilities of the network. According to a comparative experiment, FastMesh may be the best among many P2P algorithms. But this algorithm also has disadvantages. When a node enters or leaves the network, it needs to readjust the topology structure, so it is not suitable for the situation where the nodes change greatly. Mesh can also dynamically adjust the network topology according to the uplink bandwidth of each child node, so that nodes with large uplink bandwidth are closer to the source node, so as to make full use of the existing capabilities of the network. According to a comparative experiment, FastMesh may be the best among many P2P algorithms. But this algorithm also has disadvantages. When a node enters or leaves the network, it needs to readjust the topology structure, so it is not suitable for the situation where the nodes change greatly.
Our self-developed algorithm—Push-Pull algorithm combines the advantages of push-based and pull-based algorithms. It uses pull to obtain the buffer with the highest priority from the parent node, and the parent node uses push for it. Provide subsequent buffers. In addition, our algorithm mixes HTTP, HTTPS, WebRTC, Websocket and other protocols to maximize the P2P rate on the premise of ensuring user experience first. After testing, the Push-Pull algorithm has the advantages of low latency, high bandwidth utilization, high P2P rate, and strong robustness to network topology changes.
livewebPlayer
livewebPlayer is an open-source HTML5 streaming media playback framework written entirely in JavaScript. It realizes the integration of HTTP (including HTTPS, HTTP2), WebRTC's multi-protocol, multi-source, low-latency, and high-bandwidth utilization. Plug-in-free Web-side streaming media acceleration capabilities. H5-based MSE technology (Media Source Extension) feeds Buffer blocks from multiple source nodes to the player, coupled with a well-designed algorithm to achieve the optimal scheduling strategy and handle various abnormal situations, liveweb Player It can maximize the P2P rate on the premise of ensuring smooth video experience for users.
Integrating our livewebPlayer.js is also very simple. It only takes a few lines of code to import our js file into the script tag, and pass the video id and token as parameters to the function we provide.
Liveweb is a web player independently developed by Haoyou Technology. It supports RTSP, RTMP, HTTP, HLS, UDP, RTP, File and other streaming media protocols. It also has a variety of display methods (GDI, D3D) and formats (RGB24, YV12, YUY2, RGB565), after 7x24 hours of continuous copying test, it can handle disconnection very well
If you are looking for a powerful streaming media player, then liveweb will be a good choice, and we welcome your understanding and trial.
★The business card at the end of the article can receive audio and video development learning materials for free, including (FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, srs) and audio and video learning roadmaps, etc.
see below!