How to render an external X11 library into a GTK4 Widget

Platform
,
1

I’m trying to find if there’s any implementation or hack that can be done for rendering X11 content into a GTK4 child widget (not the MainWindow).

For example, if you create a Window with the following content:

 > Gtk.Window()
      > Gtk.Box()
        > Gtk.DrawingArea() 
        > Gtk.Label()

GTK3: generates a different X11 id for the Window and the DrawingArea.
GTK4: only generates an ID for the Window.

Full code snippet & explanation here.

And the problem is that many external libraries still based in X11 (instead of Wayland), so it is the only method for drawing content.

I’ve been reading posts & docs by multiple days, and it seems that GStremer archived a way of doing this by writing into a Gdk.Paintable, but I wonder if there is any other way of doing this.

In my case, I’m using VLC, so maybe I could extract the frames, and then pain them, but I wonder if there is a better solution. Also I don’t know if the performance will be too low.

  1. My first idea for hacking this, was to add a non-decorated window inside another window, but that it is not possible…

  2. The second one would be to position a non-decorated window over the main decorated window, but that seems like an horrible hack, I better stick with GTK3.

  3. Maybe something with a Gtk.Surface() ? Create a top level surface and include it in the window? is that even possible?

Do you have any idea? I see that this topic is affecting may users

Thank you,
rsm

2

I saw this recent post on Planet GNOME:

https://blogs.gnome.org/xjuan/2024/09/13/introducing-casilda-wayland-compositor-widget/

I don’t know if a XWayland window is supported to embed it.

3

Hello @swilmet ,

I did some tests and the compositor works pretty good, but sadly it does not support XWayland :confused:

4

The typical approach in a non-X11 world is to have the external library render into a shared buffer, like a GL texture, a Vulkan image, or a dmabuf. Then you take that buffer and use the GdkTexture API to let GTK render it; see:

If you are constrained by X11, you could create a Cairo surface backed by a Window or an XCB drawable, and then use the surface as the source for a drawing area; that’s going to be expensive, but it’s the cost of using legacy API.

Since you’re just rendering video, the recommendation is to use GStreamer and its paintable sink; its implementation already takes care of using the most appropriate pipeline and transport mechanism, with minimal copies and, if the stars align properly with kernel, decoding, and compositor, it can also take into account all the color space information, leading to HDR support.

1 Like
5

Hello @ebassi ,

your comments are very helpful ! I think that with vlc4 Wayland will be supported, but if it is not, I’ll probably be able to use the GL texture.

Meanwhile, before migrating all my code to GStreamer, I would like to test the X11 hack… could you please tell me if this is your idea?

I created a separate Gtk.Window(), and when it is realized, I get its surface. Then I pass the surface to the draw function of a DrawingArea, and I try to draw it:

    def __on_draw(self, _drawing_area, cairo_ctx, _width, _height):

        if self.external_surface is None:
            return

        external_cairo_surface = self.external_surface.create_similar_surface(cairo.Content.COLOR,
                                                                              self.external_surface.get_width(),
                                                                              self.external_surface.get_height())
        cairo_ctx.set_source_surface(external_cairo_surface, 0, 0)
        cairo_ctx.paint()

but the drawing are just gets black… I know that in reality I’ll have to redraw much more often (not only when I resize the window), but this should prove the idea…

Thank you,
rsm