Gstreamer in Gtk4

Platform
, ,
1

Hi ; I’ve been working on a video player, and made two prototypes:

And found out that the GtkVideo player is choking on a lot of video files, when the Gstreamer one is playing basically whatever I throw at him, so I’m going to go with it, but I want to do it in Gtk4.

Also from my trials, the Gst approach is way more flexible in terms of precise control of the video : Seeking, subtitles, audio tracks, widgets, etc.

My question is : Is this even possible? I see that Gtk.DrawingArea is still part of Gtk4 (and not flagged as deprecated) but maybe the new Paintable interface changes the way of doing this.
Also the way that the GSTreamer video draws to the DrawingArea seems to be very Gtk3-specific:

    video_window = Gtk.DrawingArea.new()
    video_window.connect("realize", self.on_realize)
    video_window.connect("draw", self.on_draw)

# this function is called when the GUI toolkit creates the physical window
# that will hold the video
# at this point we can retrieve its handler and pass it to GStreamer
# through the XOverlay interface
def on_realize(self, widget):
    window = widget.get_window()
    window_handle = window.get_xid()

(…)

    # pass it to playbin, which implements XOverlay and will forward
    # it to the video sink
    self.playbin.set_window_handle(window_handle)

But are those methods and elements available in Gtk4? I though I’d ask here is this approach is even portable to Gtk4/Adw, or if I should look right away for a new system for draw/painting my video stream to a Gtk4 widget?

To be clear, this is really my question : How would one go today about creating a Gst-based video player in Gtk4 (Adw.Application)?

I already verified that the required libs are available :

gi.require_version('Gtk', '4.0')
gi.require_version('Adw', '1')
gi.require_version('Gst', '1.0')
gi.require_version('GdkX11', '4.0')
gi.require_version('GstVideo', '1.0')

from gi.repository import Gtk, Gio, Adw, Gdk, GLib, Gst, GdkX11, GstVideo

Also, I’d like my application to be ready for Wayland, and all of this seems really XOrg specific…? I guess that’s another question: Is there a way to make a Gst-based Gtk4 video player that works on Wayland?

2

You can use gst-plugin-gtk4. That provides you with a GdkPaintable that you can place in whatever widget you want (e.g. a GtkImage) or into your own widget.

While the plugin is written in Rust, you can use it from any other language too. It works on every platform supported by GStreamer and GTK4, and will soonishly get support for direct rendering via OpenGL too.

Using the old GstVideoOverlay embedding of video into UIs is not recommended anymore and won’t very nicely with GTK4 and/or Wayland anyway.

1 Like
3

Hi Sebastian, thanks for your quick answer ;

While the plugin is written in Rust, you can use it from any other language too. It works on every platform supported by GStreamer and GTK4, and will soonishly get support for direct rendering via OpenGL too.

Wow ; Is there a Python example to illustrate such a flexibility? I’m only asking because I know very little about Rust (or even Python for that matter, I’m basically a JS / Web dev dabbling in Desktop UI) let alone Rust and Python together :slight_smile:

Using the old GstVideoOverlay embedding of video into UIs is not recommended anymore and won’t very nicely with GTK4 and/or Wayland anyway.

Yes, it does feel awkward, and I’d really like a better way ; But isn’t GtkDrawingArea a paintable itself? I’m confused as to why (and also how, see above) I would need a plugin to just get access to a standard widget?

Also, is the Cairo / GstVideoOverlay the only alternative (even in Gtk3)? What about OpenGl, like Clapper does?

EDIT : Stop the press, I found out, using another GtkVideo prototype outside of flatpak, that it can play all the problematic videos without any problem!! Exactly as Gst (yes, I have an extensive suite of tests with many weird video streams / encodings) so I’m willing to keep on trying using it. I’m going to make a topic to try and find out exactly how to make flatpak aware of the right system codecs. Clapper, on flatpak install, installs those dependancies / runtimes:

xdg-run/gvfsd, xdg-run/pipewire-0:ro, xdg-videos

Don’t I need something like that? I’m still investigating this flatpak issue (I have basically every permission on) any help appreciated.

4

I don’t have a Python example but if you write one I’d be happy to add it to the repository. It should basically work the same way: you create the pipeline and the UI, get the paintable from the sink’s property and make use of it in the GTK UI.

GtkDrawingArea is a widget that you can paint into via a callback, but also not very efficiently because it goes around cairo.

GdkPaintable is a basic interface for anything that can be painted (check the docs). GdkTexture is another example of a paintable. You can use those directly in a GtkImage and some other widgets, or when writing your own widget you can use them as part of your GtkWidget::snapshot() implementation to draw your widget. E.g. for an actual video widget you might want to draw some OSD on top of the video, which you could do inside snapshot() in addition to using the GdkPaintable you get from the gtk4paintablesink.

In GTK3 you can use gtksink / gtkglsink (from gst-plugins-good). The latter works via OpenGL and probably does more or less the same as Clapper does.

In GTK4 you either go the slow Cairo way, or do something around GdkPaintables (which then could be OpenGL-backed) inside some widget.

GstVideoOverlay generally does not work properly with Wayland, but for systems where it works it can potentially be faster as it goes around the whole GTK rendering infrastructure and can directly use hardware overlays. As a side effect you won’t be able to draw anything on top of the video though.

1 Like
5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.