Re-enabling NVIDIA DMA-buf in remote desktop

Hi all.

I’m new to Gnome development, but have a couple of performance patches to gnome-remote-desktop i’d like to see integrated. One of the patches re-inserts code that was disabled as non working, and later deleted ( Nvidia, RDP: after !3537 no windows updates visible, warning: [EGL Thread] Failed to update buffer data (#3297) · Issues · GNOME / mutter · GitLab ), but i believe i have found the underlying bug so the whole thing should be usable now.

Since it had already been deleted I didn’t want to simply request a merge without first discussing the decision to remove. That bug also mentions replacing the whole thing with Vulkan, etc, so i’m not sure if re-inserting this code is the right path forward. Can someone chime in?

I started chasing this bug because I run a headless remote desktop in a VM, with a passthrough NVIDIA GPU and just dragging windows around had gnome-shell use 80%-90% of a CPU, and gnome-remote-shell would use a bit more than that, so two full cores pegged for something that should be hardware accelerated. Post this patch the CPU usage went down to maybe 20% per process.

There are other patches that further bring down usage, and move the bottleneck to audio and encryption, to discuss after we solve this one

Well, since I got no reply i went ahead and created Enable and fix the NVIDIA dma-buf hardware encode path (#342) · Issues · GNOME / gnome-remote-desktop · GitLab , and submitted a MR to go with it. Hopefully re-introducing the old deleted code path is acceptable in order to get the huge performance benefit.