WO2011000200A1

WO2011000200A1 - Effective streaming media transmission method

Info

Publication number: WO2011000200A1
Application number: PCT/CN2009/076284
Authority: WO
Inventors: 邹联忠
Original assignee: 深圳市融创天下科技发展有限公司
Priority date: 2009-07-03
Filing date: 2009-12-30
Publication date: 2011-01-06
Also published as: CN101945427B; CN101945427A

Abstract

The present invention refers to an effective streaming media transmission method based on data transmission protocol of User Datagram Protocol (UDP). The method includes the following steps: building a video document at a server, wherein the document includes several data frames and each data frame corresponds to one frame serial number and one frame time stamp; separating each data frame into several data packets, or combining several data frames to form one data packet, wherein each data packet corresponds to a packet serial number and a packet time stamp; sending data packets to a client and storing the packet serial numbers and packet time stamps of the data packets in the queue of memory by the server; detecting by the client whether packet loss occurs, and if yes, feeding the packet serial number of the lost packet back to the server; finding out the corresponding packet time stamp according to the packet serial number in the queue by the server; finding out the data frame corresponding to the lost packet based on the relationship between a packet time stamp and a frame time stamp, separating one data frame into several data packets or combining several data frames to form one data packet, and sending the lost data packet to the client.

Description

Instruction manual

Title of Invention: An efficient streaming media transmission method

#细 or

[1] The present invention relates to the field of wireless streaming media, and in particular, to an efficient streaming media transmission method.

[2] With the rapid development of mobile communication technology and video compression methods, the use of wireless IP networks to transmit real video streaming technology has attracted people's attention. When transmitting data over a wireless IP network and a best-effort Internet, packet errors such as packet loss and overload can occur. The value of the packet loss rate is generally small, but it has a great impact on users. It is a killer that affects video quality. In particular, the compression ratio currently used is relatively high. Loss or damage of a video frame will also affect other related frames. Decompression directly damages the smoothness and clarity of the video playback at the receiving end. Therefore, error control processing for transmission at the same time of video transmission is required.

[3] Although the connection-based TCP protocol can provide reliable data transmission capabilities, its flow control mechanism is based on the low bit error rate of the wired network and low latency. As network bandwidth extension products (BDPs) increase, the usual TCP protocols become inefficient. This is because its AMDD (additive increase multiplicative decrease) algorithm completely reduces the TCP congestion window, but cannot quickly restore the available bandwidth. In addition, the unfair RTT in TCP congestion control (the round trip between R _OU nd-Trip Time packets) causes unfair sharing of bandwidth by concurrent TCP flows with different _RTTs . Although the usual TCP implementation is used in a small BDP network to achieve relatively equal shared bandwidth, in a wireless network with a large number of BDPs, concurrent TCP streams of different RTTs usually have to suffer from serious unfairness problems. In the TCP protocol, the sender manages the information of the transmitted data packet through the transmission window and the timer, and the receiving end sends back an ACK acknowledge character (ACKnowledge Character) response to the received correct data packet, and the ACK response itself will also consume a considerable amount. Network and processor resources. In addition, in the TCP protocol, for data packet retransmission, data that has been sent but not yet confirmed is stored in the TCP protocol stack. For a high-concurrency wireless streaming server, the cache for each user is extremely large. The waste of host resources is more serious in wireless networks with large network bandwidth and high bandwidth jitter rate.

[4] Based on the UDP protocol, UDP (User Datagram Protocol) User Datagram Protocol, data transmission Loss, there is no congestion window, congestion control, and data caching as described in TCP. However, the UDP protocol does not provide the error control function provided by the T CP. Nowadays, some reliable data transmission based on UDP is also to buffer the transmitted data and wait for the ACK response. This wastes resources for streaming servers with a large amount of data and large users, and reduces the amount of concurrent servers. For example, if the server sends 500 packets, the server concurrency is 8000, and the size of each packet is 1K, then the server cache needs 500x8000xlK about 4G of memory. This has a huge impact on the performance of the server.

[5] In view of this, it is necessary to propose an improved method to overcome the deficiencies of the prior art.

[6] In view of this, the present invention aims to provide an efficient streaming media transmission method based on the UDP protocol. According to the characteristics of the wireless network, an efficient streaming video service on-demand scheme is proposed, which can improve the system. The user's experience and the resources of the server itself.

[7] In order to solve the above problems, the present invention provides an efficient streaming media transmission method, and the specific steps of the method are as follows:

[8] Step S1: Establish a video file on the server, the video file includes a plurality of data frames, and each data frame corresponds to a frame serial number and a frame inter-frame stamp;

[9] Step S2: According to the length of the data frame, each data frame is divided into several data packets, or several data frames are combined into one data packet, and each data packet corresponds to one packet serial number and one packet time stamp. ;

[10] Step S3: The server sends a data packet to the client, and records the packet serial number and the packet stamp of the data packet into the queue of the memory;

[11] Step S4: The client detects whether there is a lost data packet according to the packet serial number, detects that there is a packet loss, returns the lost packet serial number to the server, and the server finds the corresponding according to the packet serial number in the queue in the memory. Baggage stamp,

[12] Step S5: According to the mapping relationship between the inter-packet stamp and the inter-frame stamp, find one or more data frames corresponding to the lost data packet in the video file, and divide one data frame into several data packets, or Several data frames form a packet and then send the lost packet to the client.

[13] The invention has the beneficial effects that the method of the invention can avoid the data cached by the server and retransmit the data that has been sent, which can greatly save the resources of the server.

Country deletion [14] The present invention will be further described below in conjunction with the accompanying drawings and embodiments, in which:

[15] FIG. 1 is a flow chart of a method for efficiently transmitting a high-efficiency streaming media according to the present invention;

[16] FIG. 2 is a diagram showing a data frame sequence number, a data frame length, and a corresponding frame inter-frame stamp of a local video file of the server;

[17] FIG. 3 is a diagram showing a packet sequence number, a packet length, and a corresponding packet stamp after the video file shown in FIG. 2 is packaged according to the method of the present invention;

[18] FIG. 4 is a queue representation diagram corresponding to the packet sequence and the inter-packet stamp stored in the memory after the server sends the data packet shown in FIG. 3;

[19] FIG. 5 is a schematic diagram of a server sending a data packet according to the present invention;

[20] FIG. 6 is a diagram showing the relationship between the memory footprint of the prior art server and the amount of users sent by the server; [21] FIG. 7 is the memory footprint and the amount of users of the method for sending the streaming media by the server. Diagram of the relationship.

difficult

[22] The present invention designs an efficient streaming media on demand scheme based on the UDP protocol, which is based on the feature of playing an on-demand file, and extracts the strategy of not caching the already transmitted data, thereby greatly saving server resources. .

[23] The specific implementation of the present invention will be described below with reference to the accompanying drawings.

[24] FIG. 1 is a schematic flow chart of the method of the present invention, and the specific steps of the method are as follows:

[25] Step S1: Establish a video file on the server, the video file includes a plurality of data frames, each data frame corresponds to a frame serial number and a frame inter-frame stamp; wherein the data frame can be a video frame or an audio

[26] As shown in FIG. 2, it is a data frame serial number, a data frame length, and a corresponding frame inter-frame stamp of the local video file of the server, wherein the first column in the figure: 1, 2, 3, ... 18 is the data frame serial number, the second column len = 2853, len=167, len=68, ... len=16 is the corresponding data frame length; the third column time=0.00000, ti time=0.24000, ...time =2.16000 is the corresponding frame inter-frame stamp. Here, the interval between frames is equal, that is, one frame is separated by 0.12000 seconds.

[27] Step S2: Dividing each data frame into several data packets according to the length of the data frame, or combining several data frames into one data packet, each data packet corresponding to a packet serial number and an inter-packet stamp ; [29] S21: First determine the packet length L in a wireless transmission environment, the size of each data frame is n, when n is greater than L吋, the frame is cut into η/L+1 data packets, before n/ Each L packet has a length of L and the last packet has a length of 0%! ^, where n/L is the rounding function after division, and % is the remainder function after division; when n is less than L吋, the data is added by the frame or subsequent data frames smaller than L to form a data length less than or equal to L data packet;

[30] S22: establishing a mapping relationship between the inter-frame stamp and the frame inter-frame stamp, when n is greater than L吋, wherein the _n /L+l records have the same inter-frame stamp, that is, the frame inter-frame stamp of the data frame; If n is smaller than L吋, the inter-packet stamp recorded by the data packet is the frame inter-frame stamp corresponding to the first data frame whose length is less than L.

[31] FIG. 3 is a diagram showing a packet sequence number, a packet length, and a corresponding packet interval after the video file shown in FIG. 2 is packaged according to the method of the present invention: First, a packet length L is set equal to 1200, the data frame length n of frame sequence 1 in Figure 2 is 2853, so n is greater than L, and the frame sequence 1 data frame is cut into n/L+1 data packets, that is, 2853/1200+1=3 data, before The length of the 2 data packets is 1200, and the length of the third data packet is ! %L, which is the remainder 453 of 2853/1200. Thus, in Figure 3, the first column is the packet length corresponding to the packet sequence numbers 1, 2, and 3. They are len=1200, len=1200, len=453, and are obtained by cutting data frame 1. Therefore, the time between the sequence numbers 1, 2, and 3 and the frame number of the frame number 1 are the same. Time=0.00000, time=0.00 000, time=0.00000;

[32] In FIG. 2, the frame sequence number 2 has a length of 167 less than L (1200), and the subsequent frame sequence number 3 has a length of 68 and is less than L, and 167 plus 68 equals 235 to satisfy less than 1200, so the frame serial number 2 3, a data packet is formed (the packet number 4 in FIG. 3, the packet length is 235, and the corresponding inter-packet stamp is the frame inter-frame stamp corresponding to the first data frame smaller than L, that is, the frame serial number 2 corresponds to Frame time stamp time=0.12000).

[33] If n is equal to L吋, the frame is treated directly as a packet.

[34] The packaging process of subsequent data is analogous to the above, and will not be mentioned here.

[35] Step S3: The server sends a data packet to the client, and records the packet serial number and the packet stamp of the data packet into the queue of the memory;

[36] S31: When n is greater than L吋, only the packet sequence number of the first packet of n/L+1 packets and the corresponding packet interval are recorded in the queue;

[37] S32: When n is smaller than L吋, the data frames in the queue that are smaller than n are added to form a packet sequence number of a data packet whose data length is less than or equal to L and the first data frame whose length is less than L. Packing time stamp. [38] Please refer to FIG. 3 and FIG. 5, the server sends a data packet to the client according to the package serial number 1, 2, 3...18 according to the packaged data packet.

[39] As shown in FIG. 4, for the information recorded in the server memory queue, the first column 1, 4, 5...18 is the packet serial number, where 1 is the third of the three packets. A packet number, 4 is the packet number of 0.12000 between packets.

[40] Step S4: The client detects whether there is a lost data packet according to the packet serial number, detects that there is a packet loss, returns the lost packet serial number to the server, and the server finds the corresponding according to the packet serial number in the queue in the memory. The package is marked as follows:

[41] S41: If the missing packet sequence number fed back by the client can be found in the queue, it indicates that the lost packet is composed of the first data packet with n greater than L吋 or several data frames with n smaller than L吋. The data packet, the server finds the packet serial number corresponding to the packet interval;

[42] In this step, for example, the lost packet sequence number of the client to feedback is 1, 4, so the packet serial number 1 and 4 can be found in the memory of the server (as shown in FIG. 4), and the corresponding package can be found respectively. Time stamp.

[43] S42: If the lost packet sequence number fed back by the client cannot be found in the queue, it indicates that this is a data packet in a data frame larger than L, and the serial number of the packet is found in the serial number queue. Between the nearest previous serial number, find the previous serial number corresponding to the packet;

[44] For example, the missing packet sequence number returned by the client is 2 or 3, so the packet serial number 2 or 3 (as shown in FIG. 4) and the corresponding packet stamp cannot be found in the memory of the server, so The previous packet sequence number 1 closest to the serial number is found in the sequence number queue, and the previous packet sequence number closest to the packet sequence number 2 or 3 is only the packet sequence number 1.

[45] Step S5: According to the mapping relationship between the inter-packet stamp and the inter-frame stamp, find one or more data frames corresponding to the lost data packet in the video file, and divide one data frame into several data packets, or Several data frames form a packet and then send the lost packet to the client.

[46] S51: If the missing packet serial number can be found in the queue, n is greater than L吋, then the corresponding frame inter-frame stamp is directly obtained from the inter-packet stamp, and the data frame in the video file is read by the inter-frame stamp. And divide the data frame into n/L+

1 data packet, resending the first data packet to the client; n is less than L吋, then the first frame length is less than L, and the frame and subsequent data less than n are found by the inter-packet stamp. Adding frames to form a data packet whose data length is less than or equal to L is resent to the client; [47] For example, if the packet serial number 1 is lost, the corresponding inter-packet stamp is 0.00000, the corresponding inter-frame stamp 0.00000 is found by the inter-band stamp 0.00000, and the data frame 1 in the video file is read by the inter-frame stamp 0.00000, and Divide the data frame into 2853/1200+1=3 packets, and resend the first packet to the client; the lost packet sequence number 4 packet is 0.12000, because the packet is the first one. A frame sequence number of less than 1200 (shown in Figure 2) is 0.12000, so a packet of length 253 and less than 1 200 consisting of frame sequence numbers 2, 3 is resent to the client.

[48] S52: If the lost packet serial number cannot be found in the queue, the data frame is cut and several data packets according to the data frame of the previous serial number corresponding to the inter-frame stamp corresponding to the inter-frame stamp. Subtracting the missing packet sequence number from the previous sequence number yields a difference _D , where (n/L+l) packets are in the (D+

1) The data packet is the lost data packet, and the (D+1) data packet is resent to the client.

[49] For example, the lost packet sequence number of the client to feedback is 2 or 3, where the previous packet sequence number closest to the packet sequence number 2 or 3 is only the packet sequence number 1, and the packet interval is 0.00000. Frame inter-frame stamp 0. 00000, frame inter-frame stamp 0.00000 reads data frame 1 in the video file, and divides the data frame into 2853/12 00+1=3 packets, which will lose the packet sequence number 2 or 3. Subtracting the previous sequence number 1 to obtain a difference D equal to 1 or 2, wherein the (D+1) of the 3 packets is the 1+1=2 or 2+1=3 packets. For the lost packet, resend the second (packet serial number 2, length 1200) or the third data packet (packet serial number 3, length 453) to the client.

[50] The method of the present invention can avoid the data cached by the server and retransmit the already sent data, which can greatly save the resources of the server. Figure 6 shows the relationship between the amount of ordinary streaming server users and memory. Figure 7 shows the relationship between the amount of streaming server users and memory after using the method described in this paper. It can be seen that after using this technology, the memory usage is significantly reduced. The amount is significantly improved. Server performance improvements are even more pronounced in wireless networks with large delays.

The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalents, and improvements made within the spirit and scope of the present invention should be included in the present invention. Within the scope of protection of the invention.

Claims

Claim

[Claim 1] 1. A streaming media transmission method, which is based on a UDP data transmission protocol, and is characterized in that it comprises the following steps:

Step S1: Establish a video file on the server, where the video file includes multiple data frames, and each data frame corresponds to a frame serial number and a frame inter-frame stamp;

Step S2: dividing each data frame into a plurality of data packets according to the length of the data frame, or combining several data frames into one data packet, each data packet corresponding to a packet sequence number and an inter-packet stamp;

Step S3: The server sends a data packet to the client, and records the packet sequence number and the packet stamp of the data packet into the queue of the memory;

Step S4: The client detects whether there is a lost data packet according to the packet serial number, detects that there is a packet loss, returns the lost packet serial number to the server, and the server finds the corresponding packet according to the packet serial number in the queue in the memory. Step S5: According to the mapping relationship between the inter-packet stamp and the inter-frame stamp, find one or more data frames corresponding to the lost data packet in the video file, and divide one data frame into several data packets, or The data frames form a data packet, and then the lost data packet is sent to the client.

[Claim 2] 2. The streaming media transmission method according to claim 1, wherein: the step S

The 1st frame inter-frame stamp is equally divided.

[Claim 3] 3,

The streaming media transmission method according to claim 2, wherein the step S2 comprises the following steps:

S21: First, determine a packet length L in a wireless transmission environment, and the size of each data frame is n. When n is greater than L吋, the frame is cut into η/L+1 data packets, and the first n/L data. Each packet has a length of L, and the last packet length is n%L, where: n/L is the rounding function after division, and % is the remainder function after division; when n is less than L吋, the frame or Subsequent data frames smaller than n are added to form a data packet whose data length is less than or equal to L; S22: Establish a mapping relationship between the inter-frame stamp and the frame inter-frame stamp, where n is greater than L吋, where the n/L+1 record has the same inter-frame stamp, that is, the frame inter-frame stamp of the data frame; when n is less than L吋, the packet interval recorded by the data packet is the frame inter-frame stamp corresponding to the first data frame whose length is less than L.

4,

The streaming media transmission method according to claim 3, wherein the step S3 comprises the following steps:

S31: When n is greater than L吋, only the packet sequence number of the first data packet of η/L+1 data packets and the corresponding packet interval are recorded in the queue;

S32: When n is smaller than L吋, a plurality of data frames smaller than n are recorded in the queue and added to form a packet sequence number of a data packet whose data length is less than or equal to L, and a packet sequence corresponding to the first data frame whose length is less than L. stamp.

5,

The streaming media transmission method according to claim 4, wherein the step S4 comprises the following steps:

S41: If the lost packet sequence number fed back by the client is found in the queue, the lost data packet is a data packet consisting of a first data packet with n greater than L吋 or several data frames with n smaller than L吋. The server finds the packet sequence number corresponding to the packet interval; S42: If the missing packet sequence number fed back by the client cannot be found in the queue, it indicates that this is a packet in a data frame larger than L, and The previous packet sequence number closest to the packet sequence number is found in the queue, and the previous sequence number corresponding to the packet is found.

6,

The streaming media transmission method according to claim 5, wherein the step S5 comprises the following steps:

S51: If the missing packet serial number can be found in the queue, n is greater than L吋, then the corresponding frame inter-frame stamp is directly obtained from the inter-packet stamp, and the data in the video file is read by the inter-frame stamp. The data packet; if n is smaller than L吋, the first frame length is less than L, and the frame and the subsequent data frames smaller than n are added to form a data length less than or equal to L. The package is resent to the client;

S52: If the lost packet serial number cannot be found in the queue, according to the data frame of the previous serial number corresponding to the inter-frame stamp corresponding to the inter-frame stamp, the data frame is cut with several data packets, and the packet is lost. The serial number is subtracted from the previous serial number to obtain a difference D, where the (D+1)th packet in the (η/L+l) data packet is the lost data packet, and the (D+1)th The packet is resent to the client.

[Claim 7] 7. The streaming media transmission method according to claim 1, wherein: the data frame may be a video frame or an audio frame.

[Claim S] 8. The streaming media transmission method according to claim 3, wherein: when n is equal to L

吋, the frame is directly used as a data packet.