Any good example of Python code for GStreamer + Wayland?

quan · May 12, 2020, 8:19am

Hi,

I want to write an application to scan QR code, based on GNOME stack: GTK+, Gstreamer, Wayland.

I plan to use Gstreamer to stream webcam video and embed in a GTK+ app. I want this app to be pure Wayland. The problem is that there are so few documentation about letting Gstreamer put its display to an area of GTK window. There is one example about GStreamer + Wayland in gstreamer-plugins-bad repo, but it is in C. I want Python + GIR.

So, could you please give a working example for Python 3.8 + Gstreamer 1.16.2?

j_arun_mani · May 13, 2020, 8:02am

Hi.
If the example you mentioned is written using GI, then porting it can be quite easy. Just look at the function used and see it’s proper binding in PyGObject API Reference.
If you run into any trouble, please feel free to bring it up ^^

quan · May 13, 2020, 10:20am

The issue is that, there is no GIR-Python equivalent for gst_wayland_display.

github.com

GStreamer/gst-plugins-bad/blob/0a2e026985d86abe92e24ef546cb81a2e4b7cc10/tests/examples/waylandsink/main.c#L95-L122


      
            display_handle = gdk_wayland_display_get_wl_display (display);
            context = gst_wayland_display_handle_context_new (display_handle);
            gst_element_set_context (GST_ELEMENT (GST_MESSAGE_SRC (message)), context);
            gst_context_unref (context);
          
            goto drop;
          } else if (gst_is_video_overlay_prepare_window_handle_message (message)) {
            GtkAllocation allocation;
            GdkWindow *window;
            struct wl_surface *window_handle;
          
            /* GST_MESSAGE_SRC (message) will be the overlay object that we have to
             * use. This may be waylandsink, but it may also be playbin. In the latter
             * case, we must make sure to use playbin instead of waylandsink, because
             * playbin resets the window handle and render_rectangle after restarting
             * playback and the actual window size is lost */
            d->overlay = GST_VIDEO_OVERLAY (GST_MESSAGE_SRC (message));
          
            gtk_widget_get_allocation (d->video_widget, &allocation);
            window = gtk_widget_get_window (d->video_widget);

This file has been truncated. show original

Can not find any GdkWaylandDisplay in https://lazka.github.io/pgi-docs.

P/S: Just found discussion about GdkWaylandDisplay: Tutorial 5: Add support for display type 'GdkWaylandDisplay' (#205) · Issues · GStreamer / gstreamer-rs · GitLab

j_arun_mani · May 13, 2020, 12:16pm

Check this guide : Implementing GStreamer Webcam(USB & Internal) Streaming[Mac & C++ & CLion] | by Edward Wang | Life’s Journey Through A Lens | Medium

It is for a completely different purpose, but should give you an idea about how the streaming works. I don’t have a webcam, so I can’t quite involve here.

ebassi · May 13, 2020, 12:23pm

GDK does not expose Wayland introspection data because the underlying C types have no introspection data either.

You can do a hacky type check of the type of GdkDisplay using a string, e.g.:

import gi
gi.require_version('Gdk', '3.0')
from gi.repository import Gdk

display = Gdk.Display.get_default()
if 'GdkWaylandDisplay' in str(type(display)):
    print('Yes')
else:
    print('No')

will print Yes when run under Wayland, and No when run under other backends.

quan · May 13, 2020, 12:51pm

Actually, I deliberately want it to work with Wayland. There is already tutorial to make it with X Window System, but I want to make a native-Wayland app.

The transition from X to Wayland is too slow, so I want to create more and more Wayland apps to urge this transition.

ebassi · May 13, 2020, 12:56pm

That’s not what drives transitions; you don’t write “Wayland” apps, you just write applications. On Wayland you can deal with some fast paths, but you’ll also need to deal with the fact that they are not available to all languages—namely, it’s mostly C.

If you want to write a video player you should really use the GTK video widget available in GStreamer; it will try and use the most appropriate pipeline for the platform it’s running on—Wayland, Windows, X11, etc—and will also use the appropriate graphics pipeline to ensure you’re not performing excessive copies of the data in order to put it on the screen.

quan · May 13, 2020, 12:56pm

My painpoint is not “how streaming work”. This tutorial missed some features, which are difficult:

Embedded the video sink to a part of existing window. That tutorial just let Gstreamer create a window on its own. It is easy.
That tutorial is for Mac, which doesn’t use Wayland. It doesn’t face the difficulty I want to overcome.

quan · May 13, 2020, 1:02pm

Could you point me the name of that feature-rich widget?
I read GStreamer documentation: Basic tutorial 5: GUI toolkit integration

but they only tell how to embed a videosink to a window, and for Linux, it is X window, not Wayland.

sdroege · May 13, 2020, 1:03pm

You can find a C example for the GStreamer gtksink and gtkglsink here: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/tree/master/tests/examples/gtk

I also have examples in Rust but I don’t know if there are examples for Python somewhere.

Generally it all works the same though: you create the sink element like any other GStreamer element, via the "widget" property you can get a GtkWidget and that you would include in your UI then. Anything passed to the GStreamer sink would then be rendered inside that GtkWidget.

sdroege · May 13, 2020, 1:14pm

Also this tutorial should really be updated to mention the GTK/QML/etc sinks. Thanks, reported that here so we don’t forget: https://gitlab.freedesktop.org/gstreamer/gst-docs/-/issues/60

quan · May 13, 2020, 4:14pm

Thank you @sdroege, trying gtksink now.

I would like to see the Rust example, too. Rust is the next language I want to play with GTK+.

sdroege · May 13, 2020, 5:15pm

You can find that here. The same directory in the repository contains lots of other more or less useful examples.

system · May 27, 2020, 5:15pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.