Major GNOME Desktop & LibAdwaita toolkit design proposal: Presentation mode/streamer mode

[TriVoxel]

Basic idea

I wish there were a simple toggle in the quick settings where the user could enable a sort of streamer/presentation mode to make it safe for them to use their desktop normally without screen recording software being able to see anything useful.

Streamer mode typically refers to a special application state, for instance in a game or chat application, where the program hides personal identifiable information so that it can be safely recorded without risk of onlookers seeing it.

Implementation

The specific implementation I had in mind for GNOME is more complex and robust. Since on Wayland we have more control over application permissions and such, would it be possible to display to the user all their personal information, such as notifications, weather/location information in the notification tray, files in Nautilus, Wi-Fi network names, and so on, but to any recording software, only display a skeleton app by giving it a separate, modified video buffer?

If that were possible, I think it could add a lot of value to the desktop, and enhance privacy & security by obfuscating personal information. This could also extend beyond streaming, and could, for example, be applied to an external monitor so that a presenter could do present things in a public meeting without the audience being able to see any confidential information if they accidentally slipped up. Because the interface remains fully in tact, even if the user enabled this on their primary monitor, the worst that could happen is they need to click the toggle off in quick settings.

Design theory

• In GNOME Control Center, there is a “Presentation mode” section where you can enable the quick setting toggle.
• Applications that define themselves as supporting the feature appear in a list with toggles to enable the effect on a per-app basis.
• LibAdwaita applications can optionally choose to implement a “Presentation mode” version of their interface, where the developer can enable it for select elements, and LibAdwaita will supply the monitor with a fully featured app, but only render text using a font like Flow Circular, use random, abstract placeholder images, and use placeholder icons, essentially like a skeleton app, which is presented to the screen recording buffer.
• This way, the user can do whatever they want, and so long as their app supports it, the audience can only see some basic shapes, and tell that the computer is being used, but not what for besides a basic layout.
• Effect is always disabled at startup

Pros

• Enhanced privacy
• A better desktop for streamers and gamers
• Benefits to professionals who present at meetings
• Easy for the user, not cluttering the quick menu by default
• Supported at the library level, making it easy for developers to implement with the LibAdwaita library doing all the heavy lifting
• User controllable
• Easily reversable
• For privacy reasons, apps and extensions should not be able to tell if streamer mode is enabled unless the setting is enabled for the app. This way, apps can’t hijak the system for DRM purposes

Cons

• This would obviously require a lot of design decisions, developer interest, and would take a lot of work to make the right adjustments to the various graphics libraries.
• This would need some kind of “warranty free” disclaimer in case of bugs
• This may introduce some slight overhead on the GPU due to rendering each applicable app twice
• This would require a lot of additional, thorough UI work and consideration
• This is very similar to a DRM system, where apps can choose to block screen recording apps, but it puts the control in the user’s hands.
• This would require new documentation

GNOME Shell issue I made (maybe not the right place?)

[jfrancis (Jason Francis)]

IMO this should not be a manual setting but something that is toggled when the screen sharing portal is used. It seems this would be used along with do-not-disturb; a simple way to start right now might be to mock up an addition to the dialog that can enable do-not-disturb mode.

[skeller (Sebastian Keller)]

Implementing this in the compositor seems like it would come with some problems:

• For this to work the size of the censored text would need to match the one of the original, but even knowing the rough length of a text can be information that can be used to significantly narrow the search space for a potential attacker (i.e. city or street names).

• If an app does not correctly mark something as private, there will be only one person who won’t be able to notice this - the streamer themselves. Because with this approach the streamer will always see the uncensored text.

Also it might not even be clear to the applications what could potentially be identifiable information. Think of things like file names in the file manager for example.

I think even providing such an option would be a disservice to the user, because we know we can’t guarantee that everything will implement support for this. Giving people the impression that this would prevent their private information from getting leaked will just prevent them from doing something that will be much more secure, like using a separate account for streaming.