OpenQA + GNOME, 2022 edition

Desktop
1

Hello,
Nearly 1 year passed since I last posted about OpenQA testing of GNOME OS.

A quick primer: OpenQA is a tool developed SuSE and used by several GNOME downstreams, including (of course) OpenSuSE. It allows us to boot a GNOME OS image in a VM, simulate clicks and key presses, and verify that the graphical output is what we expect.

GNOME’s OpenQA instance is still in a “alpha” level of deployment, you can browse it at openQA: Test results

The main announcement is that tests now live in a new GNOME/openqa-tests repo. The CI in this repo runs the tests against the last image published to https//os.gnome.org/. A full pipeline takes less than 5 minutes, so you can iterate and fix tests with a reasonable cycle time here.

image
image1182×543 63.4 KB

You can still also test specific branches of gnome-build-meta, with the test-iso-installer-x86_64 job. This job will fetch the latest commit of openqa-tests.git on startup. The cycle time is slower - you will wait 30-40 minutes at least for the iso-installer-x86_64 job to run - but it allows us to check merge requests for regressions affecting GNOME OS before landing them.

Beyond that, not much has changed, in fact things bitrotted a bit over the last year. In some recent downtime I managed to fix a few things:

This will stay in alpha status for the forseeable future unless more folk pick it up, the main blockers from my point of view to rolling this out further are:

If you’re interested in exploring OpenQA, now is a good time to play with it, and you can get some useful automated testing done as long as you understand the limitations. Some docs are written up here: https://gitlab.gnome.org/GNOME/gnome-build-meta/-/wikis/openqa/OpenQA-for-GNOME-developers

3 Likes
2

I’ve added tests for all the core apps from GNOME 43, you can see a full test run here: openQA: gnomeos-master-iso-x86_64-gnomeos@qemu_x86_64 test results

I don’t plan to add any more tests for now. Core app developers are welcome to start experimenting and adding some more complex app-specific tests.

I still consider this setup a “usable alpha”, the next thing I’m going to do is watch a weekly scheduled pipeline and see how much ongoing effort is needed to keep the tests green and useful, and see how many useful regression reports are generated.

If you want to help out, there are some open issues here: https://gitlab.gnome.org/GNOME/openqa-tests/-/issues

3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.