[Solved] Remote Desktop portal cannot simulate mouse button press on Wayland

Platform
,
1

I am writing a GTK4 application using Python which simulates user input on Wayland. On X11, this could be done using a number of ways like using pyautogui or uinput, however, on Wayland, the only workaround to make this possible without using sudo I can find is to use the Remote Desktop portal and simulate the input in a Xdp.Session, which has the benefit of needing explicit permission by the user as well.

The problem I am facing is that the calls to pointer_button() do not simulate a mouse click. There is not any output in the terminal, either.

The other methods, like pointer_motion(), pointer_position() to move the mouse and keyboard_key() to simulate keyboard key presses are working perfectly fine.

I have added “–device=all” to the Flatpak manifest. This problem does not occur when the same program is run in KDE.

How can I fix this?

2

The problem I am facing is that the calls to pointer_button() do not simulate a mouse click. There is not any output in the terminal, either.

What values to you pass to pointer_button()?

The other methods, like pointer_motion(), pointer_position() to move the mouse and keyboard_key() to simulate keyboard key presses a

Can you enable the input-events debug topic (Alt-F2 → lg → Flags → scroll down to MetaDebugTopics and toggle it.

Then run journalctl _PID=$(pgrep -x gnome-shell -u $USER) > log and reproduce the issue.

I have added “–device=all” to the Flatpak manifest. This problem does not occur when the same program is run in KDE.

This shouldn’t be needed, it doesn’t interact with devices at all.

3

Thanks for the prompt reply!

What values to you pass to pointer_button()?

Earlier I was passing 2 and Xdp.ButtonState.PRESSED, as 2 is the button number for right click in xinput. Now I am passing Gdk.KEY_Pointer_Button2 and Xdp.ButtonState.PRESSED, but the same issue is happening.

The logs are:

Jun 10 13:45:25 debian gnome-shell[2560]: INPUT: Creating new virtual input device of type 0 (0x5647ee44a1e0)
Jun 10 13:45:26 debian gnome-shell[2560]: Unknown/invalid virtual device button 0xfeea pressed
Jun 10 13:45:26 debian gnome-shell[2560]: Unknown/invalid virtual device button 0xfeea pressed
Jun 10 13:45:27 debian gnome-shell[2560]: Unknown/invalid virtual device button 0xfeea pressed
Jun 10 13:45:27 debian gnome-shell[2560]: Unknown/invalid virtual device button 0xfeea pressed
Jun 10 13:45:28 debian gnome-shell[2560]: Unknown/invalid virtual device button 0xfeea pressed
Jun 10 13:45:28 debian gnome-shell[2560]: Unknown/invalid virtual device button 0xfeea pressedGdk.KEY_Pointer_Button2, Xdp.ButtonState.PRESSED
Jun 10 13:45:29 debian gnome-shell[2560]: Unknown/invalid virtual device button 0xfeea pressed
4

The API specifies button codes should Linux evdev based: Remote Desktop - XDG Desktop Portal documentation

But it seems you are passing GDK codes. Change to use e.g. BTN_LEFT from linux/input-event-codes.h. If you’re not using C/C++, then either copy the values of the ones you care about, or see if there are dependencies you can use that packages these constants in some language specific way.

1 Like
5

Thanks! I was looking at Xdp.Session.pointer_button, which didn’t mention it had to be evdev codes. This solves my issue.

6

Feel free to open an issue on the libportal bug tracker about the documentation: Issues · flatpak/libportal · GitHub