How to simulate key press, mouse actions in a programatic way?

[j_arun_mani]

I want to make a script based on Python that can automate key press, mouse actions etc. as if they are entered manually.
Some packages or modules known : PyAutoGUI, XLib wrapper
Similar to the above, is there any package available from GI family that allows automation ? (At least that works in X11 or Wayland or GNOME desktop)

Workaround I have for now :
I managed to get windows using Wnck. Then using their XID, I got the X11 Window using GdkX11. After that I could put the events using Gdk.Display.put_event.
But the process of making Gdk.Event is confusing me. (What properties should be set, how they should be initialised etc.)

Any other module or idea or way you have for simulating actions ?
Thanks in advance

[andyholmes]

You can (mostly) use Atspi on X11, but if you need to support X11 and Wayland you’re probably restricted to the Remote Desktop portal.

If you can guarantee you’re in GNOME Shell, you might be able to use Mutter’s implementation of the remote desktop portal directly.

[StefanSalewski]

Maybe related (GUI testing):

Guides for automated testing

Is it possible to use GDK for automated testing? I need to programatically: Send key press / key release events Move the mouse, including drag and drop (click, drag and then drop/unclick) In theory I think it’s possible. I google it and did not find much material. Does any app does it? Thanks.

[snarkhunter (Steve Langer)]

See if https://www.ctcms.nist.gov/oof/gtklogger/ describes what you want. The code lets you record a session in a log file and replay it. You can add tests to the log file. You do need to modify your source code, but not drastically. See https://www.ctcms.nist.gov/oof/gtklogger/docs/gui_testing.pdf for the details.

The code on that page is only for gtk+2, but I’m currently working on a gtk3 version and would be happy to share it.

I was told on this site that the event logging approach to GUI testing was not the way to go, but I haven’t found a better solution, and this method is working for me (so far).

– Steve