Gstreamer pull WHEP form MediaMTX

[yuijzeon]

hi everybody, it bothered me for days 😣

Question

I try to use gstreamer to pull and push my streaming

I push the camera video to local MediaMTX server as test vial RTSP protocol

gst-launch-1.0 -v v4l2src ! videoconvert ! x264enc speed-preset=ultrafast tune=zerolatency ! rtspclientsink protocols=tcp location=rtsp://127.0.0.1:8554/mystream

It works very good

but when I pull streaming from MediaMTX server via WHEP protocol

gst-launch-1.0 -e whepsrc whep-endpoint=http://localhost:8889/mystream/whep ! autovideosink --gst-debug-level=3

I always stuck by "Redistribute latency..."

Setting pipeline to PAUSED ...
error: XDG_RUNTIME_DIR is invalid or not set in the environment.
Pipeline is live and does not need PREROLL ...
0:00:00.410775850   145 0x7ae0a0002920 WARN               webrtcbin gstwebrtcbin.c:3532:sdp_media_from_transceiver:<webrtcbin0> Caps application/x-rtp, media=(string)audio, encoding-name=(string)OPUS, payload=(int)96, clock-rate=(int)48000, rtcp-fb-transport-cc=(boolean)true are missing ssrc
Got context from element 'autovideosink0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayGBM\)\ gldisplaygbm0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
0:00:01.301616615   145 0x7ae08c003700 FIXME                default gstutils.c:4031:gst_pad_create_stream_id_internal:<nicesrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:01.301693583   145 0x7ae08c003700 WARN                GST_PADS gstpad.c:4361:gst_pad_peer_query:<nicesrc0:src> could not send sticky events
0:00:01.302284956   145 0x7ae0a0002920 WARN               webrtcbin gstwebrtcbin.c:3532:sdp_media_from_transceiver:<webrtcbin0> Caps application/x-rtp, media=(string)video, payload=(int)101, encoding-name=(string)VP8, clock-rate=(int)90000, rtcp-fb-nack-pli=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-transport-cc=(boolean)true; application/x-rtp, media=(string)video, payload=(int)102, encoding-name=(string)VP9, clock-rate=(int)90000, rtcp-fb-nack-pli=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-transport-cc=(boolean)true; application/x-rtp, media=(string)video, payload=(int)103, encoding-name=(string)H264, clock-rate=(int)90000, rtcp-fb-nack-pli=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-transport-cc=(boolean)true; application/x-rtp, media=(string)video, payload=(int)104, encoding-name=(string)H265, clock-rate=(int)90000, rtcp-fb-nack-pli=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-transport-cc=(boolean)true; application/x-rtp, media=(string)video, payload=(int)105, encoding-name=(string)AV1, clock-rate=(int)90000, rtcp-fb-nack-pli=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-transport-cc=(boolean)true are missing ssrc
0:00:01.510570473   145 0x7ae0a0002920 WARN               webrtcbin gstwebrtcbin.c:6671:_set_description_task:<webrtcbin0> returning error: media 0 is missing or contains an empty 'mid' attribute

The log mention "missing ssrc"

Have I missed any steps? I'm thinking maybe there is some mediaMTX config that I missed it

my mediamtx.yml is same as default setting

use browser (Chrome) I can see stream on http://localhost:8889/mystream

MediaMTX version: 1.9.3_linux_amd64

[mgdelafuente]

Try modifyng your reception pipeline to:
gst-launch-1.0 -e whepsrc whep-endpoint=http://localhost:8889/mystream/whep ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink
If still not working probably you will have to set video-caps in whepsrc.

[yuijzeon]

I've tried it, but it's still the same. It seems like the pipeline is getting stuck at the very beginning.
I also tried setting the video-caps, but I have no idea.
I tried many different parameters, but it still doesn't work. :(

[CheSauRie]

Try modifyng your reception pipeline to: gst-launch-1.0 -e whepsrc whep-endpoint=http://localhost:8889/mystream/whep ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink If still not working probably you will have to set video-caps in whepsrc.
I tried your pipeline but got the error that no such element or plugin "whepsrc" even though I installed gstreamer v1.20.3 correctly and fully.

[yuijzeon]

Try modifyng your reception pipeline to: gst-launch-1.0 -e whepsrc whep-endpoint=http://localhost:8889/mystream/whep ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink If still not working probably you will have to set video-caps in whepsrc.
I tried your pipeline but got the error that no such element or plugin "whepsrc" even though I installed gstreamer v1.20.3 correctly and fully.

There doesn't seem to be a pre-built version of this plugin

I manually wrote a script to generate and install

This is my script:
https://github.com/BeeInventor/gst-plugins-rs/blob/main/Dockerfile

[aler9]

It's tricky but i finally figured it out - basically whepsrc requires you to specify codecs in advance. Furthermore, if the audio track is not present, you still have to set audio-caps to support an empty incoming PCMU track.

Therefore, the command for reading with GStreamer and whepsrc an H264 video-only stream is:

gst-launch-1.0 whepsrc whep-endpoint=http://127.0.0.1:8889/test/whep use-link-headers=true \
video-caps="application/x-rtp,media=video,encoding-name=H264,payload=127,clock-rate=90000" \
audio-caps="application/x-rtp,media=audio,encoding-name=PCMU,payload=0,clock-rate=8000" \
! rtph264depay ! fakesink