Rtsp stream(s) to gtk4 grid

Platform
,
1

hi,

What is the simplest and most efficient way to get a gst.pipeline like playbin to render in my gtk4 GUI, preferably using gstreamer-vaapi when available?

It looks like maybe Gtk.MediaFile.new_for_input_stream, but gio.inputStream looks to be more low level than playbin.

thanks

2

I remember seeing some sample code here for how to create a gst sink that renders to a GdkPaintable:
https://gitlab.gnome.org/matthiasc/video-play

It’s some code copied out from the gtk media backend, not sure if there was ever a plan to put that in a library some place…

3

thanks! i saw that, but still haven’t buckled down and made complete sense of it yet. :slight_smile: I’ve never learned c (only what i know of it’s derivatives/successors), so i can half way understand some stuff, but the repo seems to be more complicated than i need. Also, i wasn’t sure if the code in the repo was up to date with the official gtk4 release as it was written a year ago and not updated since. It is called a playground by the author and not necessarily a current and clear cut example of the best way to do stuff.

I was hoping for a simpler example or just a high level explanation of how to use the mediastream stuff for my use case (ex. “just chain gst.this to gtk.that to gtk.that”). I’m trying to confirm what the latest, best way to do this is. Maybe mediastream is not intended to be as high level and convenient as i had hoped?

i’ve used a gtksink with gtk3 and gtk-rs a couple years ago, and it was working, but it was copying to cpu before rendering (using significantly more cpu compared to gst-launch, IIRC), and i waited for gtk4 hoping the mediastream stuff would make it avoid that cpu copy (go straight to gpu render, however that all works) and maybe even make it easier to attach a rtsp stream to a gtk container.

thanks

4

Hmm, yeah, the code in the GTK4 repo does have some more updates since then, it would make sense to rebase on that if someone was going to use it. The current code in GTK does have handling for mapping directly to a GL texture and skipping the CPU copy. I think if that was working in a convenient library form, what you would do is just create a GtkGstMediaStream connected to the GstPipeline and that would handle everything. It would probably only be a single API call.

I think part of the problem is still that:

  • GTK cannot expose this functionality because GStreamer is an optional dependency
  • GStreamer cannot add this functionality because it would create a circular dependency on GTK

So someone may just have to copy this into a separate library, I know at the very least this should be used when Cheese gets ported to GTK4.

5

I think if that was working in a convenient library form, what you would do is just create a GtkGstMediaStream connected to the GstPipeline and that would handle everything.

This is what i thought i would be able to do with gtk.MediaStream, so thanks for the clarification.

The current code in GTK does have handling for mapping directly to a GL texture and skipping the CPU copy

So is there a manual method for doing that now? IOW, do you know what gst+gtk classes and order to use, so i can investigate further and maybe get it working?

thanks again!

6

I was wrong earlier, I did a search and there are some open merge requests in gstreamer for GTK4 sinks:

7

cool. thanks for getting back. i may try to use the old gtk3 method with the lower res streams for the multi-stream grid, and the higher res stream for when a particular stream is maximized, as a workaround while this all gets sorted out for gtk4.

8

and i don’t understand what GtkGstMediaStream is for if they are updating gtksink and gtkglsink for gtk4…

9

Maybe eventually Gtk could be changed to use that sink if it gets merged? I’m not sure, I still think there could be an issue with the circular dependency there. The media modules in Gtk are for convenience – just out of curiosity, have you tried creating a GtkMediaFile with a URI set to your RTSP stream? That may just work without having to manually create the pipeline.

10

well that’s kind of where i started with the gio.inputstream, as i assumed i should use the gtk_media_file_new_for_input_stream instead of gtk_media_file_new_for_filename.

That being said, i’m trying gtk_media_file_new_for_filename , but i’m not sure how to get that attached to my grid.

11

There is some other issue with input streams and the GStreamer backend, see here: https://gitlab.gnome.org/GNOME/gtk/-/issues/4062

You might want to try something like this:

gtk_media_file_new_for_file(g_file_new_for_uri("rtsp://mystream"))

And I think that might work using rtspsrc?

12

There is some other issue with input streams and the GStreamer backend, see here: gtk_media_file_set_input_stream assertion failure when using GStreamer backend (#4062) · Issues · GNOME / gtk · GitLab

good to know, thanks.

You might want to try something like this:

gtk_media_file_new_for_file(g_file_new_for_uri("rtsp://mystream"))

Right, but my problem right now is: assuming i have a media module that works(i have one that is not throwing type errors that i want to test in my gui. see below), what is the Chain o’ Widgets to get it to render in a grid?

i have (using the nim gintro bindings):

media_file1 = gtk4.newMediaFileForFilename("rtsp://admin:passwd@192.168.1.5:554")

Gtk.Grid wants a widget when attempting to attach to it, and the media modules are not widgets. I’m guessing i’m supposed to use Gdk.Paintable somehow, but i’m not seeing how from the text on it’s api docs page…

13

You would have to use a file object with a URI, not a filename. The media stream can then be attached to a GtkVideo with set_media_stream, or to a GtkPicture with set_paintable.

1 Like
14

as an example i have this:

import gintro/[gtk4, gobject, gio, gdk4]

proc activate(app: gtk4.Application) =
  let window = newApplicationWindow(app)
  let grid = newGrid()
  let media_file1 = gtk4.newMediaFileForFile(gio.newGFileForUri("rtsp://admin:passwd@192.168.1.5:554"))  
  let picture1 = gtk4.newPicture()
  gtk4.setPaintable(picture1, cast[gdk4.Paintable](media_file1))
  window.defaultSize = (800, 600)
  grid.columnSpacing = 25 
  grid.attach(picture1, 0, -1)    
  window.setChild(grid)
  window.show

proc main =
  let app = newApplication("org.gtk.example")
  app.connect("activate", activate)
  let status = app.run
  quit(status)

main()

but it seems that the gtk4.newMediaFileForFile(gio.newGFileForUri("rtsp://admin:passwd@192.168.1.5:554"))

part is not working. I do have the real rtsp stream creds in the actual code. Everything appears to be attached properly, but no rtsp stream plays. I also tried with a gtk4.Video and set_media_stream with the same result. Any ideas?

15

I think you have to do this somewhere in there:

media_file1.play()
1 Like
16

unfortunately, that didn’t work.

17

Does it work if you play it on the command line? Something like: gst-launch-1.0 -v playbin uri=rtsp://admin:passwd@192.168.1.5:554

18

yes, it does. also, the gintro gstreamer example works here: https://github.com/StefanSalewski/gintro#a-gstreamer-example

but parseLaunch just uses it’s own window, of course.

19

Hmm, I can’t figure out what’s going on, I tried to do it using this demo rtsp stream and it seems to be playing the audio but I see no video. So it’s half working :slight_smile: but there is probably a bug somewhere and I’m not sure how to report it, maybe someone who is really good at reading gstreamer debug logs could figure something out.

I don’t have nim installed but here is my very similar javascript test program I was using:

imports.gi.versions.Gtk = '4.0';
const { Gio, Gtk } = imports.gi;

let app = new Gtk.Application({ application_id: 'com.MediaTest' });

app.connect('activate', () => {
  const win = new Gtk.ApplicationWindow({ application: app, default_width: 600, default_height: 400 });
  const file = Gio.File.new_for_uri('rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_175k.mov');
  const media = Gtk.MediaFile.new_for_file(file);
  const picture = new Gtk.Picture();
  picture.set_paintable(media);
  win.set_child(picture);
  media.play();
  win.present();
});

app.run([]);
20

thanks. i had forgotten about the rtsp stream for testing. the bigbuckbunny rtsp stream works in my example program with the media_file1.play() added (both audio and video), but my ipcam rtsp stream uri does not. the same cam uri that works with gst-launch. so, maybe the gio.newGFileForUri doesn’t like the fact that it’s not a stream of an actual file? i don’t remember/know exactly how that works under the hood, so maybe that’s fool talk. :slight_smile: