Latest understanding: these cameras are using LIVE555 Streaming Media v2013.04.08 (according to the session-level tool attribute in their DESCRIBE response) which has at least two major bugs described in live555's changelog:

2015.05.03:
- Updated the "RTSPServer" implementation to fix a bug in RTP/RTCP-over-TCP streaming.
  Before, if the "RTSPClientConnection" object closed before the "RTSPClientSession" object,
  and the TCP connection was also being used for RTP/RTCP-over-TCP streaming, then the
  streaming state (in the "RTSPClientSession") would stay alive, even though the TCP socket
  had closed (and the socket number possibly reused for a subsequent connection).
  This could cause a problem when the "RTSPClientSession" was later reclaimed (due to inactivity).
  Now, whenever a "RTSPClientConnection" object is closed (due to the RTSP TCP connection closing),
  we make sure that we also close any stream that had been using the same TCP connection
  for RTP/RTCP-over-TCP streaming.  (Thanks to Kirill Zhegulev for noting this issue.)

2013.12.04:
- Updated the "sendDataOverTCP()" function (in "RTPInterface.cpp") to allow for the possibility of
  one of the "send()" calls partially succeeding - i.e., writing some, but not all, of its data.

also, there was a follow-up to the 2015.05.03 entry two years later:

2017.06.04:
- Fixed a bug in "RTPInterface::removeStreamSocket()" that could cause not all 'TCP stream'
  records for a given socket number to be removed if a TCP socket I/O error occurred
  (during RTP/RTCP-over-TCP streaming).  (Thanks to Gerald Hansink et al for reporting this.)

Currently your best bet to successfully talk with these cameras is to ask the manufacturer for a firmware upgrade (see note below for the Reolink 420-5MP / hardware version IPC_51516M5M) or connect with UDP via retina::client::Transport::Udp (#30). We have some other ideas below.

Original comment:

As mentioned in this Apr 29th comment on a moonfire-nvr issue, Reolink cameras have an odd behavior. They sometimes set up a stream with one ssrc, eg:

RTP-Info: url=trackID=1;seq=40477;rtptime=2077624148;ssrc=c517011f,url=trackID=2;seq=0;rtptime=0;ssrc=00000000

then send RTP packets for both this one and another ssrc. I think the latter is for an earlier failed session or something. Sometimes they'll even send the RTP packets for the other ssrc before the connection is even authenticated, which seems like a security hole (if you expose the cameras to an untrusted network, which is probably a bad idea anyway).

Currently when this happens, Retina will fail with an error such as the following (from one of @jlpoolen 's logs today in this moonfire-nvr issue):

Expected ssrc=Some(21cd90d7) seq=Some(8f4a) got ssrc=6d87df1a seq=88b9 ts=1866533263 (mod-2^32: 1866533263), npt 0.000 at [192.168.1.88:33514(me)->192.168.1.48:554@2021-08-14T11:00:57 pos=1510@2021-08-14T11:00:57]

I think we should add an option to so that when we know what ssrc to expect, we just (log and) ignore packets with any other ssrc. This will likely make these cameras behave acceptably, especially if (as I suspect) the other ssrc goes away after a minute or so when the stale session times out.

My VStarcam camera is failing here:

https://github.com/scottlamb/retina/blob/main/src/client/parse.rs#L424

as you can see, it has no control attribute, but it has control attributes on the media descriptions.

sdp: SessionDescription { version: 0, origin: Origin { username: "VSTC", session_id: 3836310016, session_version: 3836310016, network_type: "IN", address_type: "IP4", unicast_address: "192.168.1.198" }, session_name: "streamed by the VSTARCAM RTSP server", session_information: None, uri: None, email_address: Some("NONE"), phone_number: None, connection_information: Some(ConnectionInformation { network_type: "IN", address_type: "IP4", address: Some(Address { address: "0.0.0.0", ttl: None, range: None }) }), bandwidth: [], time_descriptions: [TimeDescription { timing: Timing { start_time: 0, stop_time: 0 }, repeat_times: [] }], time_zones: [], encryption_key: None, attributes: [], media_descriptions: [MediaDescription { media_name: MediaName { media: "video", port: RangedPort { value: 0, range: None }, protos: ["RTP", "AVP"], formats: ["96"] }, media_title: None, connection_information: None, bandwidth: [Bandwidth { experimental: false, bandwidth_type: "AS", bandwidth: 1536 }], encryption_key: None, attributes: [Attribute { key: "control", value: Some("track0") }, Attribute { key: "rtpmap", value: Some("96 H264/90000") }, Attribute { key: "fmtp", value: Some("96 packetization-mode=1;profile-level-id=4d002a;sprop-parameter-sets=Z00AKp2oHgCJ+WbgICAoAAADAAgAAAMAfCA=,aO48gA==") }] }, MediaDescription { media_name: MediaName { media: "audio", port: RangedPort { value: 0, range: None }, protos: ["RTP", "AVP"], formats: ["8"] }, media_title: None, connection_information: None, bandwidth: [Bandwidth { experimental: false, bandwidth_type: "AS", bandwidth: 64 }], encryption_key: None, attributes: [Attribute { key: "control", value: Some("track1") }, Attribute { key: "rtpmap", value: Some("8 PCMA/8000/1") }] }] }

Here's the raw response:

RTSP/1.0 200 OK
Cseq: 2
Date: Mon, Jul 26 2021 17:35:15 GMT
Content-Type: application/sdp
Content-Length: 403

v=0
o=VSTC 3836309504 3836309504 IN IP4 192.168.1.198
s=streamed by the VSTARCAM RTSP server
e=NONE
c=IN IP4 0.0.0.0
t=0 0
m=video 0 RTP/AVP 96
b=AS:1536
a=control:track0
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=4d002a;sprop-parameter-sets=Z00AKp2oHgCJ+WbgICAoAAADAAgAAAMAfCA=,aO48gA==
m=audio 0 RTP/AVP 8	 
b=AS:64
a=control:track1
a=rtpmap:8 PCMA/8000/1

is the control really needed?

cargo run --example client mp4 --url rtsp://192.168.1.198:10554/tcp/av0_0 --username admin --password 123456 --initial-timestamp ignore /home/dev/orwell/video_samples/tmp/cam_test.mp4

gives

E20210727 19:24:02.308 main client] Fatal: [172.17.0.2:50402(me)->192.168.1.198:10554@2021-07-27T19:23:55, 119614@2021-07-27T19:24:02, channel=0, stream=0, ssrc=0000173a] packet follows marked packet with same timestamp

from here https://github.com/scottlamb/retina/blob/b1db9a9e8b94ff7077050cc26be0a50cbf1bd58e/src/codec/h264.rs#L181

I don't know why --initial-timestamp ignore is needed but without it I get

https://github.com/scottlamb/retina/blob/b1db9a9e8b94ff7077050cc26be0a50cbf1bd58e/src/client/mod.rs#L705

Anyways, the packet follows marked packet with same timestamp error occurs after around 30 calls to push, like this:

push
push
push
push
push
push
push
push
push
push
push
push
push
push
push
push
8023 (mod-2^32: 8023), npt 0.000: 111304-byte video frame
push
E20210727 19:36:04.662 main client] Fatal: [172.17.0.2:50572(me)->192.168.1.198:10554@2021-07-27T19:36:01, 113756@2021-07-27T19:36:04, channel=0, stream=0, ssrc=00001f57] packet follows marked packet with same timestamp

as you see, a video frame is generated and then we get the error. It always happens after the first video frame is emitted.

The strange thing is that on my app, the retina client does not produce this error. I'm trying to figure out what is different.

[section moved to #15]

Do you have any idea on the error on the mp4 example, and on my client?

It'd be nice to support RTP/UDP in addition to RTP/TCP (interleaved channels) for at least a couple reasons:

• counterintuitively, I think this would be more reliable with some cameras:
  • at least those affected by #17
  • even modern live555 servers because its buffer management is terrible. If a send buffer fills mid-packet, it will block the whole server on that one stream, so it might be better for live555 to not have any RTP/TCP clients at all.
• possibly lower latency, depending on network conditions and implementation. At least in theory this comes at the expense of more dropped packets.

Work for receiving data over UDP:

• separate "sessions" from "connections". They're separate types already, but they're always created and destroyed together, and I think some fields might be in the wrong place.
• manage the UDP socket(s). I think the simplest model is for each UDP session to own its UDP socket(s) (maybe just one, or maybe up to two per stream: one for RTP, one for RTCP). If a session is dropped without TEARDOWN, it gets handed off to some background task that retries the teardown periodically until success or timeout, then closes the socket(s). Note this is more IO library-specific logic. (It's possible to share UDP sockets between sessions, but I'm not seeing any real advantage right now, and it adds complexity.)
• in combination with #7: we'd need to define a trait for UDP, and we might want to take an "opener" rather than an existing TCP connection, so that we can retry opening the connection if it closes when we still want to send a TEARDOWN or other request.
• consider how to handle out-of-order packets. On a LAN, I think we might get away with just dropping them, and that might be the only situation I care about for now. But I think other software like ffmpeg implements reorder buffers: if we're waiting for packet n and n+1 shows up, queue n+1 until at least x ms pass or the queue size exceeds y bytes.
• send RTCP RR packets (reception reports) to help the server pace packets appropriately.

When sending data (a client using the ONVIF audio back channel, or a server, neither of which is implemented yet anyway), we'd also need to implement pacing.

On my Vstarcam, I'm getting https://github.com/scottlamb/retina/blob/b1db9a9e8b94ff7077050cc26be0a50cbf1bd58e/src/client/parse.rs#L505, that is:

    let rtp_info = response
        .header(&rtsp_types::headers::RTP_INFO)
        .ok_or_else(|| "PLAY response has no RTP-Info header".to_string())?;

Does it have to have an RTP-Info header? According to https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-rtsp/8432535c-ae1e-4d86-82de-29528cec1d3c,

This header is also sent by clients in the Play request when predictive stream selection is used. For more information on predictive stream selection, see section 2.2.6.10.5.

mine

PLAY rtsp://192.168.1.198:10554/tcp/av0_0 RTSP/1.0
Authorization: Digest username="admin", realm="RTSPD", nonce="kx2i6xqrqe44c86b667qy88lq1cl5670", uri="rtsp://192.168.1.198:10554/tcp/av0_0", response="a57bb4a903a2cddc2a40563b692da576"
CSeq: 4
Range: npt=0.000-
Session: 1632991860795359396
User-Agent: moonfire-rtsp test

RTSP/1.0 200 OK
Cseq: 4
Date: Mon, Jul 26 2021 19:14:55 GMT
Session: 1632991860795359396

yours

PLAY rtsp://192.168.1.198:10554/tcp/av0_0 RTSP/1.0
Authorization: Digest username="admin", algorithm="MD5", realm="RTSPD", nonce="03xah8kf4k0f228i77hp792zh70pd45q", uri="/tcp/av0_0", response="aa0fc4dbf695e0cb3be8cada1d1c7318"
CSeq: 5
Session: 4480630546592316416
User-Agent: RRTSP Client

RTSP/1.0 200 OK
Cseq: 5
Date: Mon, Jul 26 2021 19:17:16 GMT
Session: 4480630546592316416

I have a Dahua camera serving RTSP at rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0

I tried with correct username and password to connect to, it fails with authentication error.

Password are masked in above screenshot, but I can ensure the password is 100% correct. I tried to sniffer the communication between retina client and camera, and here is what I captured.

DESCRIBE rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
Accept: application/sdp\r\n
CSeq: 1\r\n
User-Agent: Retina mp4 example\r\n
\r\n

RTSP/1.0 401 Unauthorized\r\n
CSeq: 1\r\n
WWW-Authenticate: Digest realm="Login to 4K01FD2PAJ6E3FF", nonce="203c8d27b595204504dfa65fd22586bd"\r\n
\r\n


DESCRIBE rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
Accept: application/sdp\r\n
Authorization: Digest username="admin", realm="Login to 4K01FD2PAJ6E3FF", uri="rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0", nonce="203c8d27b595204504dfa65fd22586bd", response="61dd3463aff71f24ee30ca23e3eeae4a"\r\n
CSeq: 2\r\n
User-Agent: Retina mp4 example\r\n
\r\n

RTSP/1.0 401 Unauthorized\r\n
CSeq: 2\r\n
WWW-Authenticate: Digest realm="Login to 4K01FD2PAJ6E3FF", nonce="203c8d27b595204504dfa65fd22586bd"\r\n
\r\n

While the same RTSP url and credentials work fine in VLC. I also sniffered the communication between VLC and camera, the following are the captured packets.

OPTIONS rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
CSeq: 2\r\n
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)\r\n
\r\n

Response: RTSP/1.0 401 Unauthorized\r\n
CSeq: 2\r\n
WWW-Authenticate: Digest realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab"\r\n
\r\n


OPTIONS rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
CSeq: 3\r\n
Authorization: Digest username="admin", realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab", uri="rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0", response="f8d0870b5bf8db1967c1a04587c04fb2"\r\n
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)\r\n
\r\n

RTSP/1.0 401 Unauthorized\r\n
CSeq: 3\r\n
WWW-Authenticate: Digest realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab"\r\n
\r\n

DESCRIBE rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
CSeq: 4\r\n
Authorization: Digest username="admin", realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab", uri="rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0", response="f7d5cde2d331183cb1710e50d9738971"\r\n
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)\r\n
Accept: application/sdp\r\n
\r\n

RTSP/1.0 401 Unauthorized\r\n
CSeq: 4\r\n
WWW-Authenticate: Digest realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab"\r\n
\r\n

Request: OPTIONS rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
CSeq: 5\r\n
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)\r\n
\r\n

RTSP/1.0 401 Unauthorized\r\n
CSeq: 5\r\n
WWW-Authenticate: Digest realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab"\r\n
\r\n

OPTIONS rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
CSeq: 6\r\n
Authorization: Digest username="admin", realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab", uri="rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0", response="ad87c66be128206abbd2dd51fea8e563"\r\n
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)\r\n
\r\n

RTSP/1.0 200 OK\r\n
CSeq: 6\r\n
Server: Rtsp Server/3.0\r\n
Public: OPTIONS, DESCRIBE, ANNOUNCE, SETUP, PLAY, RECORD, PAUSE, TEARDOWN, SET_PARAMETER, GET_PARAMETER\r\n
\r\n

DESCRIBE rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
CSeq: 7\r\n
Authorization: Digest username="admin", realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab", uri="rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0", response="f6cbb524ce35db485250b8b88445365c"\r\n
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)\r\n
Accept: application/sdp\r\n
\r\n

RTSP/1.0 200 OK\r\n
CSeq: 7\r\n
x-Accept-Dynamic-Rate: 1\r\n
Content-Base: rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0/\r\n
Cache-Control: must-revalidate\r\n
Content-length: 630
Content-type: application/sdp
\r\n

You may notice VLC actually attempted twice. DESCRIBE(CSeq=4) and DESCRIBE(CSeq=7). The nonce username uri are the same. but their response are different. The first DESCRIBE failed and the later one succeeded.

DESCRIBE rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
CSeq: 4\r\n
Authorization: Digest username="admin", realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab", uri="rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0", response="f7d5cde2d331183cb1710e50d9738971"\r\n
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)\r\n
Accept: application/sdp\r\n
\r\n

DESCRIBE rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0\r\n
CSeq: 7\r\n
Authorization: Digest username="admin", realm="Login to 4K01FD2PAJ6E3FF", nonce="e8ca006dd506cf4fdd75de1950f5aeab", uri="rtsp://192.168.8.20:554/cam/realmonitor?channel=1&subtype=0", response="f6cbb524ce35db485250b8b88445365c"\r\n
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)\r\n
Accept: application/sdp\r\n
\r\n

I tried to compute digest using the following approach.

HA1 = MD5(username:realm:password)
HA2 = MD5(method:digestURI)
response = MD5(HA1:nonce:HA2)

And in its second attempt, VLC produced f6cbb524ce35db485250b8b88445365c for response, it seems VLS is trying an alternative approach to compute digest and then succeeded.

As discussed in #13, connecting to a VStarcam camera and feeding its frames to ffmpeg produced ffmpeg errors. From discussion there:

@lucaszanella wrote:

However, on my app, while the frame producing works, passing retina::codec::VideoFrame::data().borrow() to the ffmpeg nal units parser sometimes reject, and sometimes parse and send to the decoder, which procues

[h264 @ 0x559374083fc0] non-existing PPS 16 referenced
[h264 @ 0x559374083fc0] Invalid NAL unit 0, skipping.
[h264 @ 0x559374083fc0] Invalid NAL unit 0, skipping.
[h264 @ 0x559374083fc0] Invalid NAL unit 0, skipping.
[h264 @ 0x559374083fc0] Invalid NAL unit 0, skipping.
[h264 @ 0x559374083fc0] no frame!

Here's one VideoFrame:

[2021-07-27T19:52:20Z INFO liborwell::rtsp::retina_client] video frame: VideoFrame { timestamp: 189124 (mod-2^32: 189124), npt 2.061, start_ctx: RtspMessageContext { pos: 42441, received_wall: WallTime(Timespec { sec: 1627415540, nsec: 361449153 }), received: Instant { tv_sec: 421014, tv_nsec: 736674930 } }, end_ctx: RtspMessageContext { pos: 42441, received_wall: WallTime(Timespec { sec: 1627415540, nsec: 361449153 }), received: Instant { tv_sec: 421014, tv_nsec: 736674930 } }, loss: 0, new_parameters: None, is_random_access_point: false, is_disposable: false, data_len: 383 }

I wrote:

I haven't tried feeding video directly from retina to ffmpeg yet, but in principle it should work. The frames should be fine to pass to ffmpeg. How are you setting up the stream with ffmpeg? You'll likely need to pass it the extra_data from VideoParameters.

The log messages from ffmpeg suggest it's not seeing a valid stream—NAL unit types should never be 0, and I think it's rare for the PPS id to be 16 rather than 0. But maybe the problem is just that without the extra data, ffmpeg is expecting a stream in Annex B format, and my code is passing it instead in AVC format. (The former means that NAL units are separated by the bytes 00 00 01, and the latter means that each NAL unit is preceded by its length in bytes as a big-endian number which can be 2, 3, or 4 bytes long. My code uses 4 bytes.) If you prefer to get Annex B data, it'd be possible to add a knob to retina to tell it that. Or conversion isn't terribly difficult: you can scan through NAL units and change the prefix to be 00 00 01.

I suppose I could add a retina example that decodes with ffmpeg into raw images or something. What ffmpeg crate are you using?

When you don't get the packet follows marked packet with same timestamp error, have you tried saving a .mp4 and playing it back in your favorite video player? Does it work?

@lucaszanella wrote:

For ffmpeg I'm using https://github.com/lucaszanella/rust-ffmpeg-1 which uses https://github.com/lucaszanella/rust-ffmpeg-sys-1 (this one is not needed, I just added some vdpau linking stuff, the original could be used). I had to modify the rust-ffmpeg-1 to add support for ffmpeg's av_parser_parse2 which parses the individual nal units. The original project doe snot have this and he doesn't want to maintain. My patch is very experimental.

I haven't tried feeding video directly from retina to ffmpeg yet, but in principle it should work. The frames should be fine to pass to ffmpeg. How are you setting up the stream with ffmpeg? You'll likely need to pass it the extra_data from VideoParameters.

I've never needed to pass additional parameters to ffmpeg, just the nal units. I extracted the h264 bitstream from a big buck bunny .mp4 file and passed to ffmpeg calling av_parser_parse2 to break into individual nal units and then passed those units using avcodec_send_packet and it works. The same process is not working for retina. When my code used to be all C++, I used to pass the output of ZLMediaKit to ffmpeg in this way also and it worked.

Even though av_parser_parse2 has the option to pass pts, dts, etc, I never used but I'll read more about these parameters.

VideoParameters debug:

Some(Video(VideoParameters { rfc6381_codec: "avc1.4D002A", pixel_dimensions: (1920, 1080), pixel_aspect_ratio: None, frame_rate: Some((2, 15)), extra_data: Length: 41 (0x29) bytes
0000:   01 4d 00 2a  ff e1 00 1a  67 4d 00 2a  9d a8 1e 00   .M.*....gM.*....
0010:   89 f9 66 e0  20 20 28 00  00 03 00 08  00 00 03 00   ..f.  (.........
0020:   7c 20 01 00  04 68 ee 3c  80                         | ...h.<. }))

I've sent you a dump of the camera via email.

If you prefer to get Annex B data, it'd be possible to add a knob to retina to tell it that. Or conversion isn't terribly difficult: you can scan through NAL units and change the prefix to be 00 00 01.

do you have experience in which types the rtsp clients out there do these things? I've never took a deep look on how ZLMediaKit does, I simply used it and now I'm getting deeper into RTSP/RTP/h264/etc because rust had no rtsp clients so I had to make one.

This is how I extracted the big buck bunny to make it work:

ffmpeg -i BigBuckBunny_512kb.mp4 -vbsf h264_mp4toannexb -vcodec copy -an big_buck_bunny_1280_720.h264

as you see by h264_mp4toannexb, it's as you supposed.

May I know why you use the AVC format in your code? Isn't the Annex B proper for streaming?

Hello. To integrate Gstreamer and WebRTC into Retina, need to transfer the raw RTCP packet. Can you add SenderReport::raw(&self) -> &[u8]?

let launch = "rtpsession name=session \
    appsrc name=rtpsrc ! session.recv_rtp_sink \
    appsrc name=rtcpsrc ! session.recv_rtcp_sink \  
    session.recv_rtp_src ! rtph264depay ! h264parse ! vaapidecodebin ! autovideosink";

This is necessary for at least some versions of the open source v4l2rtspserver. Namely whatever version is on the earlier dafang-hacks build that works better than the latest main one on my 64mb RAM camera :D

T20211228 22:43:57.782 tokio-runtime-worker tokio_util::codec::framed_impl] frame decoded from buffer
D20211228 22:43:57.782 tokio-runtime-worker retina::client] ignoring interleaved data message on Rtcp channel 1 while waiting for response to PLAY CSeq 4
T20211228 22:43:57.782 tokio-runtime-worker tokio_util::codec::framed_impl] attempting to decode a frame
T20211228 22:43:57.782 tokio-runtime-worker tokio_util::codec::framed_impl] frame decoded from buffer
D20211228 22:43:57.782 tokio-runtime-worker retina::client] ignoring interleaved data message on Rtp channel 0 while waiting for response to PLAY CSeq 4
T20211228 22:43:57.782 tokio-runtime-worker tokio_util::codec::framed_impl] attempting to decode a frame
T20211228 22:43:57.782 tokio-runtime-worker tokio_util::codec::framed_impl] frame decoded from buffer
D20211228 22:43:57.782 tokio-runtime-worker retina::client::parse] PLAY response described stream rtsp://192.168.xx.yy:8554/unicast/track2 in Uninit state

From https://github.com/scottlamb/moonfire-nvr/discussions/151. When configured with username and password, Anpviz IPC-D250 reports:

W20210823 05:40:40.871 s-ChuckRear-sub moonfire_nvr::streamer] ChuckRear-sub: sleeping for Duration { secs: 1, nanos: 0 } after error: [192.168.254.254:39730(me)->192.168.254.5:554@2021-08-23T05:40:40, 0@2021-08-23T05:40:40] Unauthorized response to DESCRIBE CSeq=1: Non-digest authentication requested: Basic realm="/"

First of all thanks a lot for this amazing effort, I´m looking for a pure Rust RSTP client library for an IoT project and wondering if it´s possible to read a RTSP stream and publish it to a RTMP server. Can you please let me know if this is possible or if it's any plan to support this?

Summary

Add ability to break apart an Annex B stream sent in a FU-A. Fixes #68

Details

My V380 cam would send a FU-A after establishing RTSP connection. The FU-A was not conformant to spec.

FU-A (start) => [ sps header - sps - boundary - pps header - pps - boundary - idr header - idr ] # Annex B stream
FU-A (...) => [ sps header - idr ]
FU-A (end) => [ sps header - idr ]

Notice how all the frags have the same header (as they should be), but the start has an Annex B stream, meaning the last NAL picked from that packet is an IDR. This means the last NAL saved from first packet will be an IDR, but the next fragment will have... the header for SPS, but data for IDR, which is wrong.

It appears that the camera only does this for FU-A that has SPS & PPS. FU-As & single NAL units for other NAL types are conformant to spec.

I have only modified this logic for the FU-A flow. We can however use the start_rtp_nal, append_rtp_nal and end_rtp_nal to handle all NAL types.

Camera details

Name: V380 (It's a generic V380 outdoor camera) Firmware: HwV380E31_WF8_PTZ_WIFI_20201218 (I had asked them for a firmware update file to enable RTSP support)

Hi, I tried out your crate to check if it works with my outdoor camera. It exits out because of bad sps.

Is this a problem with how the camera implements RTSP?

Thank you

Output

I20221015 14:37:53.671 main client::mp4] Using h264 video stream
I20221015 14:37:53.686 main client::mp4] No suitable audio stream found
E20221015 14:37:55.589 main client] Fatal: bad sps

conn: 192.168.10.15:53286(me)->192.168.10.10:554@2022-10-15T14:37:53
stream: TCP, interleaved channel ids 0-1
ssrc: 00000000
seq: 00000068
pkt: 150505@2022-10-15T14:37:54

I am trying to run the mp4 sample: cargo run --package client mp4 --url rtsp://ip.address.goes.here/ --username admin --password test out.mp4

This is the output I get:

PS C:\Users\chriwil\source\repos\retina> cargo run --package client -- mp4 --url rtsp://localhost:8554/mystream out.mp4
    Finished dev [unoptimized + debuginfo] target(s) in 0.61s
     Running `target\debug\client.exe mp4 --url rtsp://localhost:8554/mystream out.mp4`
I20220810 14:36:31.369 main client::mp4] Using h264 video stream
I20220810 14:36:31.371 main client::mp4] Using mpeg4-generic audio stream (rfc 6381 codec mp4a.40.2)
1628175071 (mod-2^32: 1628175071), npt 0.140: 670-byte video frame
I20220810 14:36:31.537 main client::mp4] new video params: VideoParameters { rfc6381_codec: "avc1.4D401F", pixel_dimensions: (1280, 720), pixel_aspect_ratio: Some((1, 1)), frame_rate: Some((2, 50)), extra_data: Length: 43 (0x2b) bytes
0000:   01 4d 40 1f  ff e1 00 1c  67 4d 40 1f  e8 80 28 02   [email protected]@...(.
0010:   dd 80 b5 01  01 01 40 00  00 03 00 40  00 00 0c 83   ......@....@....
0020:   c6 0c 44 80  01 00 04 68  eb ef 20                   ..D....h..  }
E20220810 14:36:31.543 main client] Fatal: Timestamp jumped -2692 (-0.061 sec) from 2705800284 to 2705797592 (mod-2^32: 2705797592), npt -0.061; policy is to allow 0..10 sec only

conn: [::1]:52007(me)->[::1]:8554@2022-08-10T14:36:31
stream: TCP, interleaved channel ids 2-3
ssrc: 57185d5e
seq: 0000f0e9
pkt: 1762@2022-08-10T14:36:31
error: process didn't exit successfully: `target\debug\client.exe mp4 --url rtsp://localhost:8554/mystream out.mp4` (exit code: 1)

I have a IP camera that supports B-frames. Running the client example doesn't seem to support non-monotonic timestamps:

.\target\debug\examples\client.exe client mp4 test.mp4 --url "rtsp://172.25.127.123"
I20220624 13:08:23.257 main client::mp4] Using h264 video stream
I20220624 13:08:23.260 main client::mp4] No suitable audio stream found
W20220624 13:08:23.260 main retina::client] Connecting via TCP to known-broken RTSP server "GStreamer". See <https://github.com/scottlamb/retina/issues/17>. Consider using UDP instead!
3433148934 (mod-2^32: 3433148934), npt 0.000: 20801-byte video frame
I20220624 13:08:23.391 main client::mp4] new video params: VideoParameters { rfc6381_codec: "avc1.640033", pixel_dimensions: (2688, 1512), pixel_aspect_ratio: None, frame_rate: Some((540000, 27000000)), extra_data: Length: 42 (0x2a) bytes
0000:   01 64 00 33  ff e1 00 1b  67 64 00 33  ad 00 c5 30   .d.3....gd.3...0
0010:   0a 80 2f fe  59 b8 08 08  0d 28 00 20  f5 80 0c df   ../.Y....(. ....
0020:   e6 00 20 01  00 04 68 ee  3c b0                      .. ...h.<. }
3433156134 (mod-2^32: 3433156134), npt 0.080: 130-byte video frame
E20220624 13:08:23.570 main client] Fatal: Timestamp jumped -5400 (-0.060 sec) from 3433156134 to 3433150734 (mod-2^32: 3433150734), npt 0.020; policy is to allow 0..10 sec only

conn: 172.25.127.122:64589(me)->172.25.127.123:554@2022-06-24T13:08:23
stream: TCP, interleaved channel ids 0-1
ssrc: 8e3abf51
seq: 00002164
pkt: 23059@2022-06-24T13:08:23

Not sure what the warning about TCP is about

Hey @scottlamb , im here again after some time, do you remember me? (from discussions) ;).

I have some time now to start with my PoC (e.g: rtp to webrtc)

Im using retina to get the rtsp stream (without demuxed) and pasing it through a UDP socket to my webrtc server (example), this partially works (once start the stream suddenly stops without a clear reason, apparently retina stops sending packets).

I am using as a RTSP source stream the rtsp.stream service this.

the retina code is something like this:

#[tokio::main]
async fn main() -> Result<()> {
    let stop = signal::ctrl_c();
    let settings = Settings::new()?;

    let mut session = Session::describe(
        settings.get_src_url(),
        SessionOptions::default().creds(settings.get_credrentials()),
    )
    .await?;

    let onvif_stream_i = session.streams().iter().position(|stream| {
        matches!(
            stream.parameters(),
            Some(retina::codec::ParametersRef::Video(..))
        )
    });

    if let Some(stream_i) = onvif_stream_i {
        session.setup(stream_i, SetupOptions::default()).await?;

        let session = session.play(PlayOptions::default()).await?;

        tokio::pin!(session);
        tokio::pin!(stop);

        let listener = UdpSocket::bind("127.0.0.1:5005").await?;
        listener.connect("127.0.0.1:5004").await?;

        loop {
            tokio::select! {
                item = session.next() => {
                    if let Some(Ok(packet)) = item {
                        match packet {
                            PacketItem::Rtp(recived_packet) => {
                                info!("sending stream_id: {}, to: {}", recived_packet.stream_id(), "127.0.0.1:5004");
                                listener.send(recived_packet.raw()).await?;
                            },
                            _ => continue,
                        }

                    }
                },
                _ = &mut stop => {
                    break;
                },
            }
        }
    }

    Ok(())
}

I want to get some guideness about if is this the correct path to go, i was reading #58 and he wants to pass the frames using gstreamer rtph264depay, you answers him about using the demuxed() of retina to do that.

In my case using demuxed() on session doesn't works, i keep improving this PoC to maybe one day add it to the examples if you think is valuable.

Thanks for you work on this awesome crate.