Plugin development, my new plugin cannot link

Hi all,

I’ve been learning how to develop plugins to GStreamer and I’ve read all the tutorials, etc. As next step I’ve decided to write a new plugin. I went through the Bad Plug-ins list and decided to try to rewrite the hlssink2 in Rust as learning exercise. It seemed like a fun challenge and it relates to what I do for work. It all went well and I have almost translated line-by-line the plugins, my intention is to later add more configurations on this new plugin so I called it flexhlssink.

The problem I’m having now is while trying to test the flexhlssink with a real pipeline. I’ve tried multiple different ways to connect the “audio” pad in a way that the pipeline can be stopped after a short time (quick test). The code can be found here: https://github.com/rafaelcaricio/gst-flexhlssink-rs/blob/73efe8172027ff77ed092e9dc38f802490a8208a/tests/flexhlssink.rs

My latest attempt was to delay the pad linking as much as possible, but it still didn’t work.

    let audio_convert_clone = audio_convert.clone();
    let flexhlssink_clone = flexhlssink.clone();
    decodebin.connect_pad_added(move |_, pad| {
        let caps = pad.current_caps().unwrap();
        let s = caps.structure(0).unwrap();

        let audio_convert_sink_pad = audio_convert_clone.static_pad("sink").unwrap();

        if s.name() == "audio/x-raw" && !audio_convert_sink_pad.is_linked() {
            pad.link(&audio_convert_sink_pad).unwrap();

            let audio_convert_src_pad = audio_convert_clone.static_pad("src").unwrap();
            let hls_audio_pad = flexhlssink_clone.request_pad_simple("audio").unwrap();
            audio_convert_src_pad.link(&hls_audio_pad).unwrap();
        }
    });

The “audio” pad template in flexhlssink is set to Any so I don’t understand why it is not linking.

Relevant log lines:

0:00:00.080064000 40253 0x7fc5fc082a80 INFO               GST_EVENT gstevent.c:1610:gst_event_new_reconfigure: creating reconfigure event
0:00:00.080077000 40253 0x7fc5fc082a80 INFO        GST_ELEMENT_PADS gstelement.c:753:gst_element_add_pad:<flexhlssink> adding pad 'audio'
0:00:00.080095000 40253 0x7fc5fc082a80 INFO                GST_PADS gstpad.c:2378:gst_pad_link_prepare: trying to link audioconvert:src and flexhlssink:audio
0:00:00.080121000 40253 0x7fc5fc082a80 INFO               structure gststructure.c:2842:gst_structure_get_valist: Expected field 'channel-mask' in structure: audio/x-raw, rate=(int)44100, channels=(int)1;
0:00:00.080170000 40253 0x7fc5fc082a80 INFO                GST_PADS gstpad.c:2434:gst_pad_link_prepare: caps are incompatible
0:00:00.080179000 40253 0x7fc5fc082a80 INFO                GST_PADS gstpad.c:2529:gst_pad_link_full: link between audioconvert:src and flexhlssink:audio failed: no common format
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Noformat', tests/flexhlssink.rs:120:56
stack backtrace:
0:00:00.103048000 40253 0x7fc5fbd05a00 INFO        flexhlssink-test tests/flexhlssink.rs:186:flexhlssink: flexhlssink_pipeline: received buffer #1
   0: rust_begin_unwind
             at /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/panicking.rs:92:14
   2: core::option::expect_none_failed
             at /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/option.rs:1329:5
   3: core::result::Result<T,E>::unwrap
             at /Users/rafael.caricio/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/result.rs:1037:23
   4: flexhlssink::test_basic_element_properties::{{closure}}
             at ./tests/flexhlssink.rs:120:13
   5: <O as gstreamer::auto::element::ElementExt>::connect_pad_added::pad_added_trampoline
             at /Users/rafael.caricio/.cargo/git/checkouts/gstreamer-rs-79e52a2d27eb91a3/d30e626/gstreamer/src/auto/element.rs:820:13
   6: _g_cclosure_marshal_VOID__OBJECTv
   7: __g_closure_invoke_va
   8: _g_signal_emit_valist
   9: _g_signal_emit
  10: _gst_element_add_pad
  11: _gst_decode_bin_expose
  12: _expose_pad
  13: _type_found
  14: _ffi_call_unix64
  15: _ffi_call_int
  16: _g_cclosure_marshal_generic
  17: _g_closure_invoke
  18: _signal_emit_unlocked_R
  19: _g_signal_emit_valist
  20: _g_signal_emit
  21: _gst_type_find_element_sink_event
  22: _gst_pad_send_event_unchecked
  23: _gst_pad_push_event_unchecked
  24: _push_sticky
  25: _events_foreach
  26: _check_sticky
  27: _gst_pad_push_event
  28: _event_forward_func
  29: _gst_pad_forward
  30: _gst_pad_event_default
  31: _gst_pad_send_event_unchecked
  32: _gst_pad_push_event_unchecked
  33: _push_sticky
  34: _events_foreach
  35: _check_sticky
  36: _gst_pad_push_event
  37: _gst_queue_loop
  38: _gst_task_func
0:00:00.126622000 40253 0x7fc5fc082990 INFO                 basesrc gstbasesrc.c:2914:gst_base_src_loop:<audiotestsrc> pausing after gst_base_src_get_range() = eos
0:00:00.126650000 40253 0x7fc5fc082990 INFO                    task gsttask.c:312:gst_task_func:<audiotestsrc:src> Task going to paused
0:00:00.126687000 40253 0x7fc5fc082ad0 INFO                    task gsttask.c:312:gst_task_func:<app_queue:src> Task going to paused
0:00:00.126690000 40253 0x7fc5fbd05a00 INFO        flexhlssink-test tests/flexhlssink.rs:186:flexhlssink: flexhlssink_pipeline: received buffer #2
0:00:00.126734000 40253 0x7fc5fbd05a00 INFO              GST_STATES gstbin.c:2517:gst_bin_element_set_state:<flexhlssink> current PAUSED pending VOID_PENDING, desired next PAUSED

Would anyone be able to shine some light on what might be happening? I really appreciate the help.

Thank you!

This is your problem, and a few lines above it probably printed both caps in question.

My guess is that this fails because you

  • Link raw audio to the audio pad
  • The audio pad is a ghostpad that is proxying the audio_0 pad of splitmuxsink
  • splitmuxsink is proxying caps and other requirements with the muxer inside it
  • mpegtsmux only supports various compressed audio formats

In short, you’d need to add an audio (and video!) encoder before your element. You probably want to encode to MP3 with lamemp3enc or avenc_mp3, or to AAC with fdkaacenc or avenc_aac. And for video to H264 with x264enc.

If that’s not it, please provide a full log with GST_DEBUG=6.


Also feel free to submit your new plugin for inclusion to gst-plugins-rs once it is feature-equivalent (or more) as the existing hlssink2 plugin. I noticed that API-wise currently the signals are missing, for example.

1 Like

Thank you for the answer, it really helped. I did set the GST_DEBUG=6 and checking out the GStreamer source code in the version I’m running, so I could track the execution and figure out what I did wrong! :smiley:

The problem was that I didn’t use the template name for the pad to compare on request, but the name. :sweat_smile:

    fn request_new_pad(
        &self,
        element: &Self::Type,
        templ: &gst::PadTemplate,
-       name: Option<String>,
+       _name: Option<String>,
        caps: Option<&gst::Caps>,
    ) -> Option<gst::Pad> {
        let mut settings = self.settings.lock().unwrap();
-       match name.as_ref().map(|val| val.as_str()) {
+       match templ.name_template().as_ref().map(|val| val.as_str()) {

Also I could see the specific pad negotiation execution and understand what you mentioned in the caps not being compatible.

Now I can successfully link pads!!1 :rocket:
Thank you! When I get this plugin working I will be glad to contribute it back to gst-plugins-rs project! :slight_smile:

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.