i’m trying to learn Rust and Gtk and am writing an application for displaying ip cam streams, among other things. I’m wondering what is the most cpu efficient way to render a video in a gtk3 element using wayland. My video src is rtsp and right now i’m using gstreamer’s playbin with gtkglsink (https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/blob/master/examples/src/bin/gtksink.rs) to render video in a GtkBox (gtkglsink creates a GtkGlArea widget) that’s added to an GtkOverlay and the overall/total cpu usage is 10-15% per two cams, while just using gst-launch w/ playbin in the terminal uses 2.5% total cpu per 2 cams. I don’t know why yet, but as it stands right now, the former doesn’t even scale to 4 cams. Is there a more efficient way to render ipcams in gtk3 with wayland?
thanks
it seems to me (first gtk app) that gtk3 is a bit of a mine field in regards to displaying video with wayland.
the gstreamer people have been very helpful
but even in the above proposed implementation method, it evidently wouldn’t work with my gtk gui b/c i’m using GtkOverlays for my revealed btnboxes. Therefore, it seems i would still have no usable solution to play fully hardware accelerated rtsp video streams with wayland and gtk3 without changing my gui design.
Also, from a lazy user perspective, i wish it were simpler/more straightforward to implement this stuff. I just want to use playbin and tell it what container to put the video in. I want gtk/gst to handle all the rendering details in the background, in the most efficient way possible, using whatever hardware acceleration method i have available. In my current case that’s gstreamer-vaapi. I also want to be able to pass already playing videos to other gtk windows while maintaining their current playing state. I’m guessing that is already possible, but that i just haven’t figured it out yet.
It’s very frustrating that both ffmpeg and gstreamer can play video using low cpu (1-2% total cpu per 1080p rtsp stream) via sdl window for ffmpeg(ffplay) and a gtk window for gstreamer(gst-launch), but when i want to put these same video streams into my own gtk gui 4 way grid w/ GtkOverlays, it adds complexity/requires work-arounds and uses 7.5% total cpu per 1080p stream, which won’t work for a ip cam gui.
i’m still hopeful someone replies here and tells me what a dolt i am, and what method to use that meets my requirements. 
Otherwise, hopefully gtk4 and it’s GtkMediaStream will make things easier/better, but i’m a little concerned i may have to pause this app’s dev until that time comes.
thanks for reading my novel.
You cannot really compare SDL, which has zero complexity, to GTK, which adds complexity. I’m actually shocked that you’re getting 7.5% CPU, which is in the same order of magnitude as SDL’s 2%.
You’re adding four streams, in four different widgets, with layout, styling, input handling, and a couple of layers of indirection; of course you’re going to get less raw performance: all the stuff you’re getting out of GTK is not a free lunch.
Yes, if you want to create an IPcam application that has minimal overhead then I strongly suggest you use a very low overhead windowing system API, like SDL. If you need what GTK provides—a selection of different widgets, input handling, layout managers, and desktop integration—then I’m sorry, but you’ll have to deal with the cost of doing so.
thanks for the reply. that’s good to know, but it seems to me that gtk could do it <2% per cam if the OpenGL rendering method didn’t have to kick some things back out to the cpu or if there were an alternate method that can do it all with the GPU, using something like vaapi. I don’t know much about graphics i’m just trying to understand what all is going on here.
will gtk4’s GtkMediaStream be able to render using all gpu/fully hardware accelerated?
thanks
It should; ideally, it would pass the texture object from GStreamer to GTK directly, without copies.
i hope so. that would be great.
thanks
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.