All-batteries included GStreamer WebRTC producer

Related tags

Network programming rust gstreamer multimedia webrtc live-streaming
Overview

webrtcsink

All-batteries included GStreamer WebRTC producer, that tries its best to do The Right Thing™.

Use case

The webrtcbin element in GStreamer is extremely flexible and powerful, but using it can be a difficult exercise. When all you want to do is serve a fixed set of streams to any number of consumers, webrtcsink (which wraps webrtcbin internally) can be a useful alternative.

Features

webrtcsink implements the following features:

  • Built-in signaller: when using the default signalling server (provided as a python script here), this element will perform signalling without requiring application interaction. This makes it usable directly from gst-launch.

  • Application-provided signalling: webrtcsink can be instantiated by an application with a custom signaller. That signaller must be a GObject, and must implement the Signallable interface as defined here. The default signaller can be used as an example.

  • Sandboxed consumers: when a consumer is added, its encoder / payloader / webrtcbin elements run in a separately managed pipeline. This provides a certain level of sandboxing, as opposed to having those elements running inside the element itself.

    It is important to note that at this moment, encoding is not shared between consumers. While this is not on the roadmap at the moment, nothing in the design prevents implementing this optimization.

  • Congestion control: the element levarages transport-wide congestion control feedback messages in order to adapt the bitrate of individual consumers' video encoders to the available bandwidth.

  • Configuration: the level of user control over the element is at the moment quite narrow, as the only interface exposed is control over proposed codecs, as well as their order of priority, and disabling congestion control. Consult gst-inspect=1.0 for more information.

More features are on the roadmap, focusing on mechanisms for mitigating packet loss.

It is important to note that full control over the individual elements used by webrtcsink is not on the roadmap, as it will act as a black box in that respect, for example webrtcsink wants to reserve control over the bitrate for congestion control.

If more granular control is required, applications should use webrtcbin directly, webrtcsink will focus on trying to just do the right thing, although it might expose interfaces to guide and tune the heuristics it employs.

Building

Prerequisites

The element has only been tested for now against GStreamer master.

For testing, it is recommended to simply build GStreamer locally and run in the uninstalled devenv.

Make sure to install the development packages for some codec libraries beforehand, such as libx264, libvpx and libopusenc, exact names depend on your distribution.

git clone https://gitlab.freedesktop.org/meh/gstreamer/-/tree/webrtcsink
meson build
ninja -C build
ninja -C build devenv

Compiling

cargo build

Usage

Open three terminals. In the first, run:

cd signalling
python3 simple-server.py --addr=127.0.0.1 --disable-ssl

In the second, run:

cd www
python3 -m http.server

In the third, run:

export GST_PLUGIN_PATH=$PWD/target/debug:$GST_PLUGIN_PATH
gst-launch-1.0 webrtcsink name=ws videotestsrc ! ws. audiotestsrc ! ws.

When the pipeline above is running succesfully, open a browser and point it to the python server:

xdg-open http://127.0.0.1:8000

You should see an identifier listed in the left-hand panel, click on it. You should see a test video stream, and hear a test tone.

Configuration

The element itself can be configured through its properties, see gst-inspect-1.0 webrtcsink for more information about that, in addition the default signaller also exposes properties for configuring it, in particular setting the signalling server address, those properties can be accessed through the gst::ChildProxy interface, for example with gst-launch:

gst-launch-1.0 webrtcsink signaller::address="ws://127.0.0.1:8443" ..

The signaller object can not be inspected, refer to the source code for the list of properties.

Testing congestion control

For the purpose of testing congestion in a reproducible manner, a simple tool has been used, I only used it on Linux but it is documented as usable on MacOS too. I had to run the client browser on a separate machine on my local network for congestion to actually be applied, I didn't look into why that was necessary.

My testing procedure was:

  • identify the server machine network interface (eg with ifconfig on Linux)

  • identify the client machine IP address (eg with ifconfig on Linux)

  • start the various services as explained in the Usage section (use GST_DEBUG=webrtcsink:7 to get detailed logs about congestion control)

  • start playback in the client browser

  • Run a comcast command on the server machine, for instance:

    /home/meh/go/bin/comcast --device=$SERVER_INTERFACE --target-bw 3000 --target-addr=$CLIENT_IP --target-port=1:65535 --target-proto=udp

  • Observe the bitrate sharply decreasing, playback should slow down briefly then catch back up

  • Remove the bandwidth limitation, and observe the bitrate eventually increasing back to a maximum:

    /home/meh/go/bin/comcast --device=$SERVER_INTERFACE --stop

For comparison, the congestion control property can be set to disabled on webrtcsink, then the above procedure applied again, the expected result is for playback to simply crawl down to a halt until the bandwidth limitation is lifted:

gst-launch-1.0 webrtcsink congestion-control=disabled

License

All code in this repository is licensed under the MIT license.

Comments
  • Relicense to MPL in order to harmonize with gst-plugins-rs prior to inclusion

    Relicense to MPL in order to harmonize with gst-plugins-rs prior to inclusion

    Hey everybody,

    We are eventually going to upstream this element in gst-plugins-rs, and for the sake of harmonization it would be nice to first of all relicense the plugin from the MIT license to the Mozilla Public License Version 2.0.

    Please let me know if you're OK with this!

    • [x] @Hejsil
    • [x] @septatrix
    • [x] @praveenperera
    • [x] @compiaffe
    • [x] @sdroege
    • [x] @thiblahute
    • [x] @vivienne-w
    • [x] @mfadhlika
    • [x] @machenmusik
    • [x] @massis08
    • [x] @gondaruk
    • [x] @MathieuDuponchelle
    opened by MathieuDuponchelle 24
  • Front end crashes on firefox

    Front end crashes on firefox

    Summary

    When trying to start a stream, the firefox tab crashes as soon as the webrtc connection is meant to start.

    Steps to reproduce

    Follow the steps in README using a Firefox browser.

    What is the current bug behavior?

    Tab crashes.

    What is the expected correct behavior?

    Tab doesn't crash and the stream starts.

    Relevant logs and/or screenshots

    Received {"type":"peer","peerId":"8fa07323-a92e-4079-9a9b-c2ab2bf18119","ice":{"candidate":"candidate:8 1 TCP 1015022335 fe80::bcbf:795c:4b9b:6407 9 typ host tcptype active","sdpMLineIndex":0}} 
Got local description: {"type":"answer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0 4747432338464604538 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=fingerprint:sha-256 26:27:F3:F9:AD:A0:2F:F6:30:A4:E6:CE:FC:E6:FD:97:2A:A2:8C:69:4A:ED:D3:EB:81:1D:C2:73:40:4B:10:99\r\na=group:BUNDLE video0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 100\r\nc=IN IP4 0.0.0.0\r\na=recvonly\r\na=extmap:1 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=fmtp:96 max-fs=12288;max-fr=60\r\na=fmtp:100 apt=96\r\na=ice-pwd:2abe31c81f00aedb0e51e9890ee93d37\r\na=ice-ufrag:7a0da430\r\na=mid:video0\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP9/90000\r\na=rtpmap:100 rtx/90000\r\na=setup:active\r\na=ssrc:2137346384 cname:{4a269b7f-996e-461a-9071-a715fba29cc0}\r\n"} 
Object { peer_connection: RTCPeerConnection, ws_conn: WebSocket, peer_id: "8fa07323-a92e-4079-9a9b-c2ab2bf18119", our_id: "1854c30a-e939-408c-9efc-9690f777a30c", closed_callback: session_closed(peer_id), data_channel: null, input: null, getVideoElement: getVideoElement(), resetState: resetState(), handleIncomingError: handleIncomingError(error), … }
Sending SDP answer
Received {"type":"peer","peerId":"8fa07323-a92e-4079-9a9b-c2ab2bf18119","ice":{"candidate":"candidate:13 1 UDP 1679819007 213.5.4.15 37090 typ srflx raddr 192.168.0.101 rport 37090","sdpMLineIndex":0}}
Received {"type":"peer","peerId":"8fa07323-a92e-4079-9a9b-c2ab2bf18119","ice":{"candidate":"candidate:14 1 TCP 847249663 213.5.4.15 9 typ srflx raddr 192.168.0.101 rport 9 tcptype active","sdpMLineIndex":0}}
Received {"type":"peer","peerId":"8fa07323-a92e-4079-9a9b-c2ab2bf18119","ice":{"candidate":"candidate:15 1 TCP 843055359 213.5.4.15 47091 typ srflx raddr 192.168.0.101 rport 47091 tcptype passive","sdpMLineIndex":0}}
Received {"type":"peer","peerId":"8fa07323-a92e-4079-9a9b-c2ab2bf18119","ice":{"candidate":"candidate:16 1 UDP 1679819263 213.5.4.15 37097 typ srflx raddr 172.30.0.1 rport 37097","sdpMLineIndex":0}} 
Received {"type":"peer","peerId":"8fa07323-a92e-4079-9a9b-c2ab2bf18119","ice":{"candidate":"candidate:17 1 TCP 847249919 213.5.4.15 9 typ srflx raddr 172.30.0.1 rport 9 tcptype active","sdpMLineIndex":0}} 
Received {"type":"peer","peerId":"8fa07323-a92e-4079-9a9b-c2ab2bf18119","ice":{"candidate":"candidate:18 1 TCP 843055615 213.5.4.15 55879 typ srflx raddr 172.30.0.1 rport 55879 tcptype passive","sdpMLineIndex":0}}

    out.webm

    Desktop

    OS: Linux x86_64

    Browser: Firefox

    This has been reproduced on more than one machine tho.

    opened by mikolajkniejski 18
  • Panics when trying to use VAAPI plugins

    Panics when trying to use VAAPI plugins

    After obtaining the vappi plugins (GstVaapiEncodeVP8, GstVaapiEncodeH264 etc.) the server panics upon an incoming connection.

    The last log output from gstreamer before it dies is: webrtcsink plugins/src/webrtcsink/imp.rs:2598:webrtcsink::webrtcsink::imp:<rswebrtcsink0> applying default configuration on encoder Element { inner: ObjectRef { inner: 0x7f67d40f05a0, type: GstVaapiEncodeVP8 } }

       0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic
   3: webrtcsink::webrtcsink::imp::VideoEncoder::bitrate
             at ./plugins/src/webrtcsink/imp.rs:514:18
   4: webrtcsink::webrtcsink::imp::Consumer::connect_input_stream
             at ./plugins/src/webrtcsink/imp.rs:1130:57
   5: webrtcsink::webrtcsink::imp::WebRTCSink::on_remote_description_set
             at ./plugins/src/webrtcsink/imp.rs:1819:25
   6: webrtcsink::webrtcsink::imp::WebRTCSink::handle_sdp::{{closure}}
             at ./plugins/src/webrtcsink/imp.rs:1981:21
   7: gstreamer::promise::Promise::with_change_func::trampoline
             at /home/septatrix/.cargo/registry/src/github.com-1ecc6299db9ec823/gstreamer-0.18.6/src/promise.rs:67:13
   8: <unknown>
   9: <unknown>
  10: g_main_context_dispatch
  11: <unknown>
  12: g_main_loop_run
  13: <unknown>
  14: <unknown>
  15: start_thread
  16: __GI___clone3

    Log output from GST_PLUGIN_PATH=$PWD/target/debug GST_DEBUG="webrtc*:9,vaapi*:9" RUST_BACKTRACE=1 gst-launch-1.0 videotestsrc ! webrtcsink &>webrtc.log

    opened by septatrix 15
  • Server doesn't fully end a broken connection

    Server doesn't fully end a broken connection

    If the client is unexpectedly disconnected from the signalling server, the session on the server will remain open until the stream is finished. This can be especially problematic on low-powered devices, which could have significant performance issues when wasting processing power and memory.

    Steps to reproduce:

    1. Run the demo
    2. Add many (roughly 10 to 20 streams) to see a noticeable difference in processor usage
    3. Close the browser window or refresh the page
    4. Even after a long time, gst-launch will keep using resources
    5. When the stream ends, many end_session function calls from webrtcsink_signalling::handlers can be seen in the debug terminal

    My guess would be that the session isn't correctly ended in this case, however I do not have any Rust experience, so it's hard to pinpoint the exact source of the issue.

    opened by charodziej 14
  • Allow passing free form metadata about consumers/producers/listenners

    Allow passing free form metadata about consumers/producers/listenners

    In some applications people might need to be able to pass more metadata than the only "display-name" this commit allow that by adding a field in all signalling structure where it is free form json.

    On the "webrtcsink" side this commit adds a meta property as a gst::Structure which will be passed as a json string to the signalling server so we enforce data to be structured.

    opened by thiblahute 12
  • readme: add gst-launch with meta example

    readme: add gst-launch with meta example

    please, and thank you. I'm new to gstreamer and can't really figure out how meta works with gst-launch, nor has Google turned up anything helpful yet.

    opened by colemickens 9
  • Limited numbers of customers for a stream

    Limited numbers of customers for a stream

    i tried 2 version of your code. firstly, i used code in main branch. I did the same as your tutorial, and then i opened the web, i clicked many times (maybe 6-10 times) on stream's peer-id, and the problem appeared like the below picture,

    Screenshot from 2022-10-06 16-56-04

    some consumers is black.

    secondly, i used code in fix-multiple-vpenc. i did and clicked the same i did in the first example and it can load more consumers (maybe 30) and then consumers after that still get black like the picture.

    can you explain what happen? How can i solve the problem?

    opened by NanNguyen01 7
  • Running into issues after installing omx

    Running into issues after installing omx

    Hello, I was using webrtcsink (hash 55d30db53bb3931f6477b6c1bad4de2a5ec5f7e4) for a while in the Raspbery Pi, with the following pipeline, and it was working fine (and it is still does work on my development machine):

    v4l2src device=/dev/video3 \
    ! video/x-h264,width=320,height=240,framerate=15/1 \
    ! decodebin3 ! videoconvert ! webrtcsink \
        stun-server=stun://0.0.0.0:3478 \
        turn-server=turn://user:[email protected]:3478 \
        signaller::address=ws://0.0.0.0:6021/ \
        video-caps=video/x-h264 \
        display-name=\"/dev/video3 - H264 USB Camera: USB Camera\" \
        congestion-control=0 \
        do-retransmission=false \
        do-fec=false \
        enable-data_channel_navigation=false

    However, if I install the OMX encoders (which works fine for encoding h264), it doesn't work, showing the following error:

    root@blueos:~# GST_DEBUG=3 gst-launch-1.0 v4l2src device=/dev/video3     ! video/x-h264,width=320,height=240,framerate=15/1     ! decodebin3 ! videoconvert ! webrtcsink         stun-server=stun://0.0.0.0:3478         turn-server=turn://user:[email protected]:3478         signaller::address=ws://0.0.0.0:6021/         video-caps=video/x-h264         display-name=\"/dev/video3 - H264 USB Camera: USB Camera\"         congestion-control=0         do-retransmission=false         
do-fec=false         enable-data_channel_navigation=false
0:00:00.261615000  4359   0xf44d60 WARN      GST_PLUGIN_LOADING gstplugin.c:505:gst_plugin_register_func: plugin "/usr/local/lib/gstreamer-1.0/libwebrtcsink.so" has unknown license "MIT"Setting pipeline to PAUSED ...Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...Setting pipeline to PLAYING ...New clock: GstSystemClock0:00:00.309164271  4359   0xf3b478 FIXME             decodebin3 gstdecodebin3.c:1143:update_requested_selection:<decodebin3-0> Implement EXPOSE_ALL_MODE
0:00:00.345123958  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:842:gst_v4l2_buffer_pool_start:<v4l2src0:pool0:src> Uncertain or not enough buffers, enabling copy threshold0:00:01.477050312  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 0 is not free0:00:01.478125468  4359   0xf3b478 FIXME             decodebin3 gstdecodebin3-parse.c:433:unblock_pending_input:<decodebin3-0> Re-use existing input streams if/when possible0:00:01.480217031  4359   0xf3c170 FIXME             decodebin3 gstdecodebin3.c:1648:get_output_for_slot:<decodebin3-0> emit autoplug-continue
0:00:01.480427500  4359   0xf3c170 FIXME             decodebin3 gstdecodebin3.c:1651:get_output_for_slot:<decodebin3-0> Handle EXPOSE_ALL_MODE0:00:01.538979322  4359   0xf3b478 WARN                 v4l2src gstv4l2src.c:1123:gst_v4l2src_create:<v4l2src0> lost frames detected: count = 1 - ts: 0:00:01.2098567490:00:01.539841979  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 1 is not free0:00:01.599575156  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 2 is not free
0:00:01.663749218  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 3 is not free
0:00:01.723812343  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 0 is not free
0:00:01.783803072  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 1 is not free
0:00:01.847968333  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 2 is not free
0:00:01.908052395  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 3 is not free
0:00:01.971569947  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 0 is not free
0:00:02.031572916  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 1 is not free
0:00:02.095605520  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 2 is not free
0:00:02.155570989  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 3 is not free
0:00:02.219586926  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 0 is not free
0:00:02.279587083  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 1 is not free
0:00:02.343555208  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 2 is not free0:00:02.403549583  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 3 is not free
0:00:02.467572916  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 0 is not free
0:00:02.527541301  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 1 is not free
Redistribute latency...
0:00:02.540752707  4359 0x75464c90 WARN                omxvideo gstomxvideo.c:223:gst_omx_video_find_nearest_frame:<omxh264dec-omxh264dec0> Difference between ts (0:00:01.276523000) and frame 2 (0:00:01.269844666) seems too high (0:00:00.006678334)
0:00:02.591476822  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 2 is not free
0:00:02.593909322  4359 0x75464c90 WARN                omxvideo gstomxvideo.c:223:gst_omx_video_find_nearest_frame:<omxh264dec-omxh264dec0> Difference between ts (0:00:01.269844000) and frame 3 (0:00:01.333824760) seems too high (0:00:00.063980760)
0:00:02.640075832  4359 0x70b80b00 FIXME                default gstutils.c:4025:gst_pad_create_stream_id_internal:<videotestsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:02.651880624  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 3 is not free
0:00:02.654896978  4359 0x75464c90 WARN                omxvideo gstomxvideo.c:223:gst_omx_video_find_nearest_frame:<omxh264dec-omxh264dec0> Difference between ts (0:00:01.333824000) and frame 4 (0:00:01.393853177) seems too high (0:00:00.060029177)
0:00:02.715983905  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 0 is not free
0:00:02.718308541  4359 0x75464c90 WARN                omxvideo gstomxvideo.c:223:gst_omx_video_find_nearest_frame:<omxh264dec-omxh264dec0> Difference between ts (0:00:01.393853000) and frame 5 (0:00:01.457830916) seems too high (0:00:00.063977916)
0:00:02.769337239  4359 0x6fc43058 WARN                GST_CAPS gstpad.c:5757:pre_eventfunc_check:<capsfilter4:sink> caps video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)baseline, level=(string)4, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)15/1, interlace-mode=(string)progressive, colorimetry=(string)2:4:5:1, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, parsed=(boolean)true, codec_data=(buffer)01428028ffe1000d2742802895a0507ec04078913501000528ce025c80 not accepted
0:00:02.769760728  4359 0x6fc43058 WARN                GST_CAPS gstpad.c:5757:pre_eventfunc_check:<capsfilter4:sink> caps video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)baseline, level=(string)4, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)15/1, interlace-mode=(string)progressive, colorimetry=(string)2:4:5:1, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, parsed=(boolean)true, codec_data=(buffer)01428028ffe1000d2742802895a0507ec04078913501000528ce025c80 not accepted
0:00:02.769977082  4359 0x6fc43058 WARN                GST_CAPS gstpad.c:5757:pre_eventfunc_check:<capsfilter4:sink> caps video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)baseline, level=(string)4, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)15/1, interlace-mode=(string)progressive, colorimetry=(string)2:4:5:1, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, parsed=(boolean)true, codec_data=(buffer)01428028ffe1000d2742802895a0507ec04078913501000528ce025c80 not accepted
0:00:02.770125624  4359 0x6fc43058 WARN                GST_CAPS gstpad.c:5757:pre_eventfunc_check:<capsfilter4:sink> caps video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)baseline, level=(string)4, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)15/1, interlace-mode=(string)progressive, colorimetry=(string)2:4:5:1, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, parsed=(boolean)true, codec_data=(buffer)01428028ffe1000d2742802895a0507ec04078913501000528ce025c80 not accepted
0:00:02.770782812  4359 0x6fc43058 WARN             omxvideoenc gstomxvideoenc.c:1795:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> error: Internal data stream error.
0:00:02.770832655  4359 0x6fc43058 WARN             omxvideoenc gstomxvideoenc.c:1795:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> error: stream stopped, reason not-negotiated
0:00:02.775555051  4359   0xf3b478 WARN          v4l2bufferpool gstv4l2bufferpool.c:477:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 1 is not free
0:00:02.776771874  4359 0x6fc205c0 WARN              webrtcsink plugins/src/webrtcsink/imp.rs:2107:webrtcsink::webrtcsink::imp:<rswebrtcsink0> Codec discovery pipeline failed: Internal data stream error.
0:00:02.776912603  4359 0x6fc205c0 ERROR             webrtcsink plugins/src/webrtcsink/imp.rs:2221:webrtcsink::webrtcsink::imp:<rswebrtcsink0> error: No caps found for stream video_0
ERROR: from element /GstPipeline:pipeline0/RsWebRTCSink:rswebrtcsink0: There is no codec present that can handle the stream's type.
Additional debug info:
plugins/src/webrtcsink/imp.rs(2222): webrtcsink::webrtcsink::imp (): /GstPipeline:pipeline0/RsWebRTCSink:rswebrtcsink0:
Failed to look up output caps: No caps found for stream video_0
Execution ended after 0:00:02.504602811
Setting pipeline to NULL ...
0:00:02.777954062  4359 0x75464c90 WARN                omxvideo gstomxvideo.c:223:gst_omx_video_find_nearest_frame:<omxh264dec-omxh264dec0> Difference between ts (0:00:01.457830000) and frame 6 (0:00:01.517852291) seems too high (0:00:00.060022291)
0:00:02.785024062  4359   0xf44d60 WARN             omxvideodec gstomxvideodec.c:418:gst_omx_video_dec_shutdown:<omxh264dec-omxh264dec0> Output buffers haven't been freed; still owned downstream?
Freeing pipeline ...

    About the OMX:

    root@blueos:~# gst-inspect-1.0 omx
Plugin Details:
  Name                     omx
  Description              GStreamer OpenMAX Plug-ins
  Filename                 /usr/local/lib/gstreamer-1.0/libgstomx.so
  Version                  1.20.2
  License                  LGPL
  Source module            gst-omx
  Source release date      2022-05-02
  Binary package           GStreamer OpenMAX Plug-ins
  Origin URL               Unknown package origin

  omxanalogaudiosink: OpenMAX Analog Audio Sink
  omxh263dec: OpenMAX H.263 Video Decoder
  omxh264dec: OpenMAX H.264 Video Decoder
  omxh264enc: OpenMAX H.264 Video Encoder
  omxhdmiaudiosink: OpenMAX HDMI Audio Sink
  omxmjpegdec: OpenMAX MJPEG Video Decoder
  omxmpeg2videodec: OpenMAX MPEG2 Video Decoder
  omxmpeg4videodec: OpenMAX MPEG4 Video Decoder
  omxtheoradec: OpenMAX Theora Video Decoder
  omxvc1dec: OpenMAX WMV Video Decoder
  omxvp8dec: OpenMAX VP8 Video Decoder

  11 features:
  +-- 11 elements

    Any thoughts? Thanks!

    opened by joaoantoniocardoso 7
  • Built Rust code fails when trying to use webrtcbin pad but gst-inspect can find the plugin

    Built Rust code fails when trying to use webrtcbin pad but gst-inspect can find the plugin

    Hi,

    I've got a nix-shell with gstreamer plugins available. I'm able to get this code to build, but as soon as a consumer is added, the gst-launch pipeline that is producing video, fails inside webrtcsink code:

    thread 'async-std/runtime' panicked at 'called `Option::unwrap()` on a `None` value', plugins/src/webrtcsink/imp.rs:1201:14

    It seems like this is because it's unable to create a webrtcbin pad. But I'm not sure why. In the same nix-shell I can run:

    [cole@slynux:~/code/webrtcsink]$ gst-inspect-1.0 webrtcbinFactory Details:
  Rank                     primary (256)
  Long-name                WebRTC Bin
  Klass                    Filter/Network/WebRTC
  Description              A bin for webrtc connections
  Author                   Matthew Waters <[email protected]>

Plugin Details:
  Name                     webrtc
  Description              WebRTC plugins
  Filename                 /nix/store/81afg749765nad7xl53s9qsmvxr0sdzx-gst-plugins-bad-1.20.1/lib/gstreamer-1.0/libgstwebrtc.so
  Version                  1.20.1
  License                  LGPL
  Source module            gst-plugins-bad
  Source release date      2022-03-14
  Binary package           GStreamer Bad Plug-ins source release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBin
                         +----GstWebRTCBin

Implemented Interfaces:
  GstChildProxy

Pad Templates:
  SINK template: 'sink_%u'
    Availability: On request
    Capabilities:
      application/x-rtp
    Type: GstWebRTCBinPad
    Pad Properties:
      transceiver         : Transceiver associated with this pad
                            flags: readable

    Any hints?r

    opened by colemickens 7
  • Add support for VAAPI h.264 and VP8 encoder

    Add support for VAAPI h.264 and VP8 encoder

    This fixes #50. I am not sure what the effect of the other options in configure_encoder is so other improvements for tweaking latency etc might be possible. Additionally I added a reason inside the unreachable! macro though I guess semantically a panic or whatever mechanism gstreamer-rs exposes to signal an error would be better.

    opened by septatrix 7
  • move consumer-added before pipeline starts playing

    move consumer-added before pipeline starts playing

    As described in https://github.com/centricular/webrtcsink/issues/25#issuecomment-1048465601

    Thanks - what I discovered is that things seem to be happy if create-data-channel happens before the pipeline starts playing, but consumer-added only occurs after the pipeline is playing, and that seems to create a race condition that is easily reproducible with localhost connections. Moving consumer-added to just before pipeline starts playing eliminates the issue even with localhost connections.

    opened by machenmusik 7
Owner
Centricular
GStreamer, Multimedia and Graphics Expertise
Centricular
GitHub
Painless peer-to-peer WebRTC networking for rust wasm

Matchbox Painless peer-to-peer WebRTC networking for rust wasm applications. The goal of the Matchbox project is to enable udp-like, unordered, unreli

Johan Klokkhammer Helsing 363 Jan 5, 2023
A library for easily creating WebRTC data channel connections in Rust

Cyberdeck A library for easily creating WebRTC data channel connections in Rust.

RICHΛRD ΛNΛYΛ 34 Nov 10, 2022
A multiplayer web based roguelike built on Rust and WebRTC

Gorgon A multiplayer web-based roguelike build on Rust and WebRTC. License This project is licensed under either of Apache License, Version 2.0, (LICE

RICHΛRD ΛNΛYΛ 2 Sep 19, 2022
Easy-to-use wrapper for WebRTC DataChannels peer-to-peer connections written in Rust and compiling to WASM.

Easy-to-use wrapper for WebRTC DataChannels peer-to-peer connections written in Rust and compiling to WASM.

null 58 Dec 11, 2022
An asynchronous dumb exporter proxy for prometheus. This aggregates all the metrics and exposes as a single scrape endpoint.

A dumb light weight asynchronous exporter proxy This is a dumb lightweight asynchronous exporter proxy that will help to expose multiple application m

Dark streams 3 Dec 4, 2022
Batteries included command line interfaces.

CLI Batteries Opinionated batteries-included command line interface runtime utilities. To use it, add it to your Cargo.toml [dependencies] cli-batteri

Remco Bloemen 15 Jan 4, 2023
First class, scalable rust project generator with batteries included.

amble First class, scalable rust project generator with batteries included. Amble is in beta ??. Install | User Docs | Crate Docs | Reference | Contri

refcell.eth 13 Oct 23, 2023
A Rust based simple cli to delete all files included in `.gitignore` file from your repo

clean-local A simple CLI to remove all the files and folders included in the .gitignore file. Written in RUST Supports glob and wildcard paths in .git

Vilva Athiban P B 5 Jul 8, 2023
GStreamer HTTP Live Streaming Plugin

A highly configurable GStreamer HLS sink plugin. Based on the hlssink2 element. The flexhlssink is written in Rust and has various options to configure the HLS output playlist generation.

Rafael Carício 13 Oct 25, 2021
This crate bridges between gstreamer and tracing ecosystems.

This crate provides a bridge between gstreamer and the tracing ecosystem. The goal is to allow Rust applications utilizing GStreamer to better integra

Standard Cognition OSS 17 Jun 7, 2022
Connect GStreamer pipelines to Jitsi Meet conferences

gst-meet: Integrate Jitsi Meet conferences with GStreamer pipelines Note: gst-meet is in an alpha state and is under active development. The command-l

AVStack 41 Nov 10, 2022
GStreamer plugin for speech to text using the Vosk Toolkit.

This project has moved upstream to the GStreamer Rust Plugins: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/729 Vosk Speec

Rafael Carício 7 Apr 12, 2022
The missing batteries of Rust

stdx - The missing batteries of Rust New to Rust and don't yet know what crates to use? stdx has the best crates. Current revision: stdx 0.119.0-rc, f

Brian Anderson 1.9k Jan 8, 2023
The batteries for core that you never knew you needed.

core+ The batteries for core that you never knew you needed: core+ coreplus documentation Core+ contains types that make it possible to write no_std l

Lachlan Sneff 12 May 11, 2022
Rust-battery - Rust crate providing cross-platform information about the notebook batteries.

battery Rust crate providing cross-platform information about the notebook batteries. Table of contents Overview Supported platforms Install Examples

svartalf 326 Dec 21, 2022
Rust bindings for libjuice. Look at datachannel-rs if you need more batteries.

Rust bindings for libjuice. Look at datachannel-rs if you need more batteries.

Vyacheslav S. Troshin 3 Sep 25, 2022
A lock-free multi-producer multi-consumer unbounded queue.

lf-queue A lock-free multi-producer multi-consumer unbounded queue. Examples [dependencies] lf-queue = "0.1" Single Producer - Single Consumer: use lf

Pierre Brouca 2 Sep 11, 2022
Fast multi-producer, multi-consumer unbounded channel with async support.

Hyperbridge Fast multi-producer, multi-consumer unbounded channel with async support. Inspired by crossbeam unbounded channel. Examples Hyperbridge::c

Anton 1 Apr 20, 2022
A single-producer single-consumer Rust queue with smart batching

Batching Queue A library that implements smart batching between a producer and a consumer. In other words, a single-producer single-consumer queue tha

Roland Kuhn 2 Dec 21, 2021
command line tools for coprolite research (paleontology and archaeology): estimate the producer's body mass based on coprolite diameter by the use of regression models

OVERVIEW OF COPROSIZE coprosize employs power, exponential and cubic regression models allowing to estimate the producer's body mass based on coprolit

Piotr Bajdek 7 Nov 25, 2022