Persistent remote desktop access API

[ErikJensen]

@halfline

Well I personally wouldn’t be thrilled with seeing GDM get in the session management business. It currently starts one process (the session leader) and expects systemd and that leader to do the rest.

Sure. I wasn’t meaning to suggest GDM would start the agent in the user’s login session. Rather, I was thinking for the specific case of remoting the greeter, where there’s not a standard autostart mechanism, and even if GDM switches using DynamicUser, other login managers might not.

For this case, the GDM wouldn’t need any additional session management functionality to invoke the agent directly. It would just need to provide it to the greeter session, which would be the party responsible for starting it. (This would work because, unlike the user session, the greeter session could be relied upon to be running a GNOME/GDM-greeder session that understands launching the user agent.)

The actual user session could still use any of the several mechanisms contemplated previously.

@halfline

I get where you’re coming from, as a 3rd party isv, you want to support as much of the linux landscape as you can. I don’t know if you’ll be able to come up with a generic solution that everyone gets on-board with or not.

Obviously, there will always be niche use cases we can’t support, but I’m trying to consider the following main use cases when thinking about a future common API for remote desktop usage:

1. GDM + GNOME (our first priority)
2. SDDM + KDE Plasma
3. LightDM + Xfce (Xfce hasn’t migrated to Wayland yet, but work looks to be in progress and the result seems likely to be wlroots-based.)

While I can’t guarantee that all three of these will choose to implement (or accept implementations of) whatever we come up with, I’d to avoid any decisions that would make it especially onerous for any of them.

Additionally, there are three more use cases that would be nice to support, as long as it doesn’t negatively impact the other three use cases:

4. Headless X11 sessions
5. Simple sway session with ly on the local console and a separate login manager for remote logins.
6. A system running OpenRC

My plan for now is to create a working proof-of-concept with the existing unstable GNOME APIs, and then use the experience to inform a draft proposal for a standard API.

---

Not touched on at all yet in this discussion is audio, which is needed for accessibility, and needs to avoid accidentally playing to the local speakers when a session is headless or curtained for remote access.

Does GNOME Remote Desktop currently provide any support for audio? If so, how does redirecting the audio work? I believe when I looked into it last (over a year ago, at least), there was a suggestion that one approach would be to have a way to tag a virtual sink as a remote desktop virtual output, and have WirePlumber know to automatically route all sound to it when such a sink is present, but I don’t know if there’s been any progress in that regard.

(A general discussion of audio remoting might make more sense in PipeWire venue, but I’m curious on GNOME’s take.)

[ErikJensen]

Looking through the GRD code, there is audio support, but, as far as I can tell, it is limited to audio mirroring scenarios. It looks like it works by capturing the audio going to the existing PipeWire sinks and forwarding it via RDP, rather than setting up a virtual sink and directing output streams to it (which makes sense in the context of GRD’s display-mirroring user service mode).

Is there currently any provision when GDM launches a headless session via the GRD system service to ensure that the user’s audio is also headless, or will sounds still play on the local and remote systems simultaneously?

[ErikJensen]

FWIW, I asked on the WirePlumber bug tracker, and received the following reply:

I think this can be implemented by adding logic in the linking scripts of WirePlumber to consider some specially designated source/sink as a “curtaining” source/sink. The designation can be done with a node property that the remote desktop daemon would be responsible for setting on these nodes. As long as these nodes are present, WirePlumber will then follow that logic and redirect streams there.

So, basically what I recalled, but apparently would be much easier to implement now than it would have been two years ago.

As far as integration with GRD / CRD, I can see two possible approaches:

1. GDM or the session creates the appropriately tagged virtual audio sink at the point the session starts or transitions to being headless, and removes it when the user logs in locally. The remote desktop tool gets it audio by monitoring this output devices, similar to what GRD does today for mirrored sessions.
2. The remote desktop tool itself creates the virtual audio sink when the user connects, and removes it when it detects that the user has logged in locally or connected via a different remote desktop tool. (While it would be easier just to remove the sink on disconnect, this runs the risk of leaking audio (potentially annoying housemates/coworkers or leaking sensitive information) if the user gets disconnected unexpectedly.