How to simulate key press, mouse actions in a programatic way?

Platform Core
1

I want to make a script based on Python that can automate key press, mouse actions etc. as if they are entered manually.
Some packages or modules known : PyAutoGUI, XLib wrapper
Similar to the above, is there any package available from GI family that allows automation ? (At least that works in X11 or Wayland or GNOME desktop)

Workaround I have for now :
I managed to get windows using Wnck. Then using their XID, I got the X11 Window using GdkX11. After that I could put the events using Gdk.Display.put_event.
But the process of making Gdk.Event is confusing me. (What properties should be set, how they should be initialised etc.)

Any other module or idea or way you have for simulating actions ?
Thanks in advance :smiley:

1 Like
2

You can (mostly) use Atspi on X11, but if you need to support X11 and Wayland you’re probably restricted to the Remote Desktop portal.

If you can guarantee you’re in GNOME Shell, you might be able to use Mutter’s implementation of the remote desktop portal directly.

1 Like
3

Maybe related (GUI testing):

1 Like
4

See if gtklogger describes what you want. The code lets you record a session in a log file and replay it. You can add tests to the log file. You do need to modify your source code, but not drastically. See https://www.ctcms.nist.gov/oof/gtklogger/docs/gui_testing.pdf for the details.

The code on that page is only for gtk+2, but I’m currently working on a gtk3 version and would be happy to share it.

I was told on this site that the event logging approach to GUI testing was not the way to go, but I haven’t found a better solution, and this method is working for me (so far).

– Steve

1 Like
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.