I'm receiving rtp packets with JPEG payload with VLC. When I mannually setup the type to 26 (JPEG), vlc doesn't try open the stream; if I define it as 96, vlc opens it and displays it wrong - that is due to malformed encoding.
To find out the correct encoding, i.e to find out what values of packet headers are correct, i want to compare the rtp packets with working example. Surprisingly, example uses payload type 96 instead of 26. I user wireshark to observe the headres; if works fine with payload type 26 RTp/JPEG, but I can't force it to decode 96-type rtp packet as 26-type rtp packet to observe JPEG headers of the working example; how can I do this? May be there are other tools to browse PEg/RTP headers without any coding?
Payload type 26 is MJPEG and wireshark parses these packets like a charm.
Payload type more than 96 are dynamic types, there is no reason to stream MJPEG with type 96. What is the source of media? Did you check the type of stream there?
Probably type 96 means h263 or h264 video stream, many cameras do like this. As far as I know, Wireshark cannot decode h264 header/payload.
Related
I set up a video stream and captured its packets(H264 over RTP). Looking at the Wireshark capture(decoded with type 96), I need to figure out the format of the GOP and the length of it. Problem is I can't tell which frame is I/P/B. Can I do this by looking at the Wireshark capture or do I need some sort of extension?
While you can get the NAL unit type of each frame quite easily by looking at the H.264 RTP payload format, I would recommend rather using a tool such as ffprobe to do the work for you:
ffprobe -show_frames -rtsp_transport tcp "<rtsp URI>" | grep -E 'pict_type'
which will output something like
pict_type=I
pict_type=P
pict_type=P
pict_type=P
In my example I use an RTSP stream but you should be able to adapt this to the RTP stream.
The input raw bitstream has type 9 NALs (AU delimiter).
There are multiple NALs of type 5 or 1 between NAL type 9. for ex: {NAL-9, NAL-5, NAL-5, NAL-9, NAL-1, NAL-1, NAL-9, NAL-1, NAL-1 ....}. I believe the encoder used slice mode (raw stream is from windows h.264 encoder which has slice mode enabled via low latency flag).
How do i feed such stream to iOS?
My guess: put all NALs between two NAL-9 into one buffer, removed NAL-9, and send those buffers to HW decoder.
In my example, buffers going to decoder will look like : {NAL-5, NAL-5}, {NAL-1, NAL-1}, {NAL-1, NAL-1}. The output result shows that decode happens but with lot of green artifacts which makes me think that there is something missing in the above approach.
My work is like the following:
Before streaming a video, the type, size and start address of each NAL unit is extracted using H.264 bitstream parser.
During streaming, a single NAL unit is encapsulated into one RTP packet.
My question: I need to use the extracted NAL unit's information as input to a mux so this mux could determine whether an RTP packet contains a specific NAL unit type such as PPS. If so, it will steer it to a TCP tunnel else, the RTP packet will be steered to a UDP tunnel.
FYI : I'm using OEFMON which integrates Qualnet and Directshow
Any help will be appreciated.
The rtp header does not have information about the NAL type. You have to parse the RTP payload to get the nal type.
The following code snippet shows the basics:
nType = getbits(pRaw+12, 3, 5);
where pRaw is the start of the entire RTP packet, which makes pRaw + 12 the start of the RTP payload. So the function essentially reads the value defined by the 5 bits starting at offset 3 from the beginning of the RTP payload data.
This is defined in RFC 6184.
I have the pcap of a VoIP call involving SILK.
I'm able to see in Wireshark the RTP payload.
From the RTP headers I can understand the sample rate (e.g. 24 KHz) and the frame size (e.g. 20 ms).
What I'd like to do is extract the RTP payload and generate a file containing the SILK-encoded audio.
From the RTP payload format description I can see that in the case of storage in a file, each block of audio needs a block header, to specify the sample rate and block size (because the block size is variable and can be different on each frame).
How can I generate a file with the correct file header ("magic number") and add a block header for each block of audio?
I can use a few different programming languages so I'm mainly interested in the required algorithm, but would appreciate references to code implementations (or perhaps some existing tool?).
Use pcaputil of pjproject: Converts captured RTP packets in PCAP file to WAV file or play it to audio device. Can filter the PCAP file on source or destination IP or port, is able to deal with SRTP and supports all codecs in PJMEDIA, including SILK (have not tried this myself).
Examples:
pcaputil file.pcap output.wav
pcaputil -c AES_CM_128_HMAC_SHA1_80 -k VLDONbsbGl2Puqy+0PV7w/uGfpSPKFevDpxGsxN3 file.pcap output.wav
Usually for videos the marker bit of RTP Packet indicates the last packet of the RTP.
So, with this it is guaranteed that I will receive 1 frame per packet or can receive more than one?
In the case beyond the depacketization I would have to make a parser to separate the H.264 frames?
If I can get more than one frame per RTP packetit is possible to get a piece of the next frame? Or all frames within the RTP packet even if more than one are completes?
Best regards,
RFC 6184 "RTP Payload Format for H.264 Video" has answers for the raised questions. It can be both ways: 2+ NAL units per packet, and 1 NAL unit fragmented over 2+ packets.
See quotes below:
5.7.1. Single-Time Aggregation Packet (STAP)
A single-time aggregation packet (STAP) SHOULD be used whenever NAL
units are aggregated that all share the same NALU-time.
and
5.8. Fragmentation Units (FUs)
This payload type allows fragmenting a NAL unit into several RTP
packets. Doing so on the application layer instead of relying on
lower-layer fragmentation (e.g., by IP) has the following advantages: