谈一谈 Live555 SERVER RTP+NALU 组包过程

其他 2018-08-28 14:42:58 阅读次数: 0
版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/engineer_james/article/details/81810023

前言

live555 处理 请求消息 四 “PLAY” 续 中已经写了start stream 流程，但是没有专门写RTP 包完整的组成过程
start stream

当我写完流程图，自己都吓了一跳，这么复杂，所以准备后面每个部分拆分

下面组包的前提，已经读取150000个字节在内存中，在谈一谈 live555 (*.mkv) track1 和 track2 的信息获取，已经读取了. 具体 MatroskaFileParser::parseEBMLIdAndSize 方法中用

而读取 内存的指针变量是 curBank() 

void StreamParser::ensureValidBytes1(unsigned numBytesNeeded) {
//....
  // Try to read as many new bytes as will fit in the current bank:
  unsigned maxNumBytesToRead = BANK_SIZE - fTotNumValidBytes;
  fInputSource->getNextFrame(&curBank()[fTotNumValidBytes],
                 maxNumBytesToRead,
                 afterGettingBytes, this,
                 onInputClosure, this);
}
unsigned char* curBank() { return fCurBank; }

知道数据的来源后，就可以解析RTP包的数据

RTP Header 组成

void MultiFramedRTPSink::buildAndSendPacket(Boolean isFirstPacket) {
  nextTask() = NULL;
  fIsFirstPacket = isFirstPacket;

  // Set up the RTP header:
  unsigned rtpHdr = 0x80000000; // RTP version 2; marker ('M') bit not set (by default; it can be set later)
  rtpHdr |= (fRTPPayloadType<<16);
  rtpHdr |= fSeqNo; // sequence number
  fOutBuf->enqueueWord(rtpHdr);

  // Note where the RTP timestamp will go.
  // (We can't fill this in until we start packing payload frames.)
  fTimestampPosition = fOutBuf->curPacketSize();
  fOutBuf->skipBytes(4); // leave a hole for the timestamp

  fOutBuf->enqueueWord(SSRC());

  // Allow for a special, payload-format-specific header following the
  // RTP header:
  fSpecialHeaderPosition = fOutBuf->curPacketSize();
  fSpecialHeaderSize = specialHeaderSize();
  fOutBuf->skipBytes(fSpecialHeaderSize);

  // Begin packing as many (complete) frames into the packet as we can:
  fTotalFrameSpecificHeaderSizes = 0;
  fNoFramesLeft = False;
  fNumFramesUsedSoFar = 0;
  packFrame();
}

上面的代码在第一次开始打包的时候，isFirstPacket = true,

所以
unsigned rtpHdr = 0x80000000; RTP Header 根据RTP协议规定 V=2 占用2bit
rtpHdr |= (fRTPPayloadType<<16); 左移16bit 表示负载 96
rtpHdr |= fSeqNo; 序列号

在初始化RTPSink的时候已经初始化 序列号 时间戳 还有SSRC 

  fSeqNo = (u_int16_t)our_random();
  fSSRC = our_random32();
  fTimestampBase = our_random32();

最后放入 fOutBuf->enqueueWord(rtpHdr); buffer中

fOutBuf->skipBytes(4); 给时间戳流出4个字节的位置
fOutBuf->enqueueWord(SSRC()); 将SSRC 填充到buffer中

因为第一次 fOutBuf->curPacketSize() = 0 所以后面直接跟 NALU
代表的偏移 fCurOffset

接下来就是打包一帧

NALU Indicator & NALU Header

void H264or5Fragmenter::doGetNextFrame() {


  if (fNumValidDataBytes == 1) { //第一次读取数据 需要从文件中读取 数据
    fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1,
                   afterGettingFrame, this,
                   FramedSource::handleClosure, this);
  } else { 
  //已经读取了数据 但是根据读取的NALU 第一个字节和第二个字节判断 是NALU 单一模式 还是 分片NALU
  //这里分成三种1. NALU 单一模式 2. 分片NALU FU-A start 3. FU-A 和 FU-end
  //概念在：
  //https://blog.csdn.net/engineer_james/article/details/81750864
  //https://blog.csdn.net/engineer_james/article/details/81782780
  //https://blog.csdn.net/engineer_james/article/details/81782824
  //https://blog.csdn.net/engineer_james/article/details/81783014
  //https://blog.csdn.net/engineer_james/article/details/81783188
  //https://blog.csdn.net/engineer_james/article/details/81783517


    fMaxSize = fMaxOutputPacketSize;

    fLastFragmentCompletedNALUnit = True; // by default
    if (fCurDataOffset == 1) { // case 1 or 2
      if (fNumValidDataBytes - 1 <= fMaxSize) { // case 1  NALU 单一模式 只有NALU Header
    memmove(fTo, &fInputBuffer[1], fNumValidDataBytes - 1);
    envir() <<"NAL unit data exist case 1 fTo = "<< fTo<<" fTo[0] = "<<fTo[0] <<" fInputBuffer[1] = "<< fInputBuffer[1] <<" \n";

      } else { // case 2 FU-A start FU indicator+FU header

    if (fHNumber == 264) {
      fInputBuffer[0] = (fInputBuffer[1] & 0xE0) | 28; // FU indicator
      fInputBuffer[1] = 0x80 | (fInputBuffer[1] & 0x1F); // FU header (with S bit)
    }
    memmove(fTo, fInputBuffer, fMaxSize);
    envir() << "NAL unit data exist case 2 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1] = " << fTo[1] << " fInputBuffer[1] = " << fInputBuffer[1] << " \n";

      }
    } else { // case 3 FU-A & Fu-A end     FU indicator+FU header

      unsigned numExtraHeaderBytes;
      envir() << "fCurDataOffset =  " << fCurDataOffset  << " \n";

      if (fHNumber == 264) {
    fInputBuffer[fCurDataOffset-2] = fInputBuffer[0]; // FU indicator
    fInputBuffer[fCurDataOffset-1] = fInputBuffer[1]&~0x80; // FU header (no S bit)
    numExtraHeaderBytes = 2;
      } 
      //根据发送包的大小拆分， 判断是中间的FU-A 还是 如果最后包大小小于最大允许发送大小就是 FU-A end 
      if (numBytesToSend > fMaxSize) {
            // We can't send all of the remaining data this time:
            numBytesToSend = fMaxSize;
            fLastFragmentCompletedNALUnit = False;
      } else {
            // This is the last fragment:
            fInputBuffer[fCurDataOffset-1] |= 0x40; // set the E bit in the FU header
            fNumTruncatedBytes = fSaveNumTruncatedBytes;
      }
      memmove(fTo, &fInputBuffer[fCurDataOffset-numExtraHeaderBytes], numBytesToSend);
      envir() << "NAL unit data exist case 3 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1] = " << fTo[1] << " fInputBuffer["<<fCurDataOffset-numExtraHeaderBytes<<"] = " << fInputBuffer[fCurDataOffset - numExtraHeaderBytes] << " \n";


    }
 //.......
    FramedSource::afterGetting(this);
  }
}

这里就是处理NALU header 地方并且针对不同的情况加了log

    //第一种情况 NALU 单一模式 不需要分片 只有8个字节的NALU Header
    envir() <<"NAL unit data exist case 1 fTo = "<< fTo<<" fTo[0] = "<<fTo[0] <<" 
    fInputBuffer[1] = "<< fInputBuffer[1] <<" \n";

    //第二种情况 NALU 需要分片  分片FU-A start  所以会有FU indicator + header 2个字节
    envir() << "NAL unit data exist case 2 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1] 
    = " << fTo[1] << " fInputBuffer[1] = " << fInputBuffer[1] << " \n";

    //第三种情况 NALU 需要分片  分片FU-A 和 FU-A end 唯一能区分就是包是否小于可允许最大size 如果是 那就是最后一个FU-A，反之分包并发送中间的FU-A
      envir() << "NAL unit data exist case 3 fTo = " << fTo << " fTo[0] = " << fTo[0] << " 
      fTo[1] = " << fTo[1] << " fInputBuffer["<<fCurDataOffset-numExtraHeaderBytes<<"] = " <<
       fInputBuffer[fCurDataOffset - numExtraHeaderBytes] << " \n";

参考 log如下

若只要fTo[0] 就是NALU Header 值，有fTo[0] 和fTo[1]的值，表示 FUindicator 和 FU header
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 103 fInputBuffer[1] = 103 //SPS
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 104 fInputBuffer[1] = 104 //PPS
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 6 fInputBuffer[1] = 6   //SEI
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 101 fInputBuffer[1] = 101 //IDR
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65 //NON-IDR
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65 //NON-IDR
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
//FU-A start 
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
//FU-A end
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[2884] = 92
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[1442] = 92
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[1442] = 92
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[2884] = 92
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[2884] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[4326] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[5768] = 92
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[2884] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[4326] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[5768] = 92
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[2884] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[4326] = 92
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 28 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 28 fTo[1] = 65 fInputBuffer[1442] = 28
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[2884] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[4326] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[5768] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[7210] = 92
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[2884] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[4326] = 92
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 28 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 28 fTo[1] = 65 fInputBuffer[1442] = 28
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 28 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 28 fTo[1] = 1 fInputBuffer[1442] = 28
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 28 fTo[1] = 65 fInputBuffer[2884] = 28
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[2884] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[4326] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[5768] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[7210] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[8652] = 92
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[10094] = 92


....

上面都是根据 fMaxSize 判断 是否应该使用分片NALU ,fMaxSize = 1444 ,计算过程如下 1456-12(RTP hdr) ourMaxPacketSize() = 1456 

fOurFragmenter = new H264or5Fragmenter(fHNumber, envir(), fSource, OutPacketBuffer::maxSize,ourMaxPacketSize() - 12/*RTP hdr size*/);

//OutPacketBuffer::maxSize = 60000
setPacketSizes((RTP_PAYLOAD_PREFERRED_SIZE), (RTP_PAYLOAD_MAX_SIZE));
RTP_PAYLOAD_MAX_SIZE = 1456

具体的参数计算 可能需要自己加log 计算后面需要详细知道参数获取过程

还需要学习MKV协议,才能解析具体读出来的参数意义

猜你喜欢

转载自blog.csdn.net/engineer_james/article/details/81810023
今日推荐
NetBSD 禁止提交由 AI 生成的代码
Apache Doris 2.0.10 版本正式发布！
开源日报 | 大模型开战；大模型独角兽被曝卖身；周鸿祎建议谷歌开源所有产品；最大开源AI社区提供1000万美元共享GPU
开源日报 | Chrome内置Gemini的意义不在于Gemini；中国AI追随之路的五大误区；ECharts创始人“下海”养鱼；谷歌I/O开发者大会什么都有，只是没有惊喜
微软回应中国区AI团队“打包赴美”传闻
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
周排行
static方法和非static方法的区别（java）
如何查找计算机专业paper
java.lang.ClassFormatError: Incompatible magic value 0 in class file com/sitecha
跳跃游戏II
stm32_之【建立工程】
TeaWeb v0.0.9 发布，统计底层优化、主机监控功能改进
事件分发 -----控制字体大小
JavaScript DOM练习（动态表格添加） December 25，2019
JSF Scope & CDI
实现从零搭建一个登录注册页面（附源代码）
每日归档
更多
2024-05-19(0)
2024-05-18(4)
2024-05-17(34)
2024-05-16(6)
2024-05-15(24)
2024-05-14(0)
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022