Feedback requested: WGAF, early GNOME Wayland automation tooling

Development
, ,
1

Hi everyone,

I’m developing WGAF (Wayland GNOME Automation Framework), a command-line automation project for GNOME on Wayland:

https://github.com/Ranrar/wgaf

WGAF currently combines:

  • A GNOME Shell extension for window and workspace operations
  • AT-SPI for inspecting and interacting with accessible application UI
  • Linux uinput for keyboard and pointer input
  • A daemon with per-capability permission policies and an emergency input stop

The intent is to make GNOME desktop automation possible without X11 compatibility layers or attempts to bypass Wayland’s security model.

Project status: WGAF is experimental and not production-ready. It has no stable release, some behaviour is still flaky across real desktop sessions, and I am actively working through reliability and safety gaps.

In particular, synthesized keyboard and pointer input currently goes to whichever window has focus at that moment. That makes it unsuitable for unattended or safety-critical automation today; preventing unintended delivery is planned work.

The project already has commands for window management, keyboard and pointer input, and AT-SPI UI discovery and interaction. It also includes deterministic GTK test applications for exercising those paths, but I would not yet describe it as a ready-made CI solution.

I would value feedback from GTK and GNOME developers on a few questions:

  • What are the hardest parts of testing GTK applications through AT-SPI today?
  • Which accessibility metadata or events are missing, unstable, or difficult for automation tools to use well?
  • Are there established GNOME testing tools, APIs, or practices WGAF should learn from before it grows further?
  • Would stable, semantic UI automation be useful in your development workflow? What constraints would it need to satisfy?
  • What would make a tool like this trustworthy enough to consider for automated testing in a GNOME Wayland session?

Critical feedback is welcome, especially around the architecture, security model, and whether this addresses a real gap. I’m looking for direction early, not presenting WGAF as a finished solution.

Thanks for your time.