The GNOME Foundation has been sponsoring a hosted macOS CI runner for a few years, but the admins are now planning to retire it for a couple of reasons:
the hardware is getting long in the tooth (it’s still an x86_64 machine)
it’s a shared bare metal environment, for licensing purposes (you can only virtualise macOS on an Apple machine running macOS), so CI pipelines compete for resources and tend to clobber each other unless the dependencies are shared or are completely isolated and never installed system-wide
nobody in the infrastructure team really knows how to handle the quirks of macOS
Additionally, the maintainers of GLib and GTK are not familiar with macOS, and nobody has volunteered to consistently monitor and handle CI issues on that platform—unlike Linux and Windows.
The result is that, more often than not, the macOS CI jobs in GLib and GTK have been blocking the development of the project.
As a result, unless somebody decides to volunteer to own the macOS CI pipeline, and work on improving it, the infrastructure team is going to retire the macOS CI builder; macOS support will go back to being best effort, and both GLib and GTK will rely on downstream packagers and consumers to test changes.
If you are experienced in managing a GitLab CI pipeline on macOS, and you wish to volunteer for this particular task, please join the #gtk:gnome.org or the #infrastructure channels on Matrix, to coordinate with GLib and GTK maintainers, and with the GNOME infrastructure team. The deadline is going to be the release of GNOME 45, September 16, 2023.
Unfortunately I don’t have the time to maintain the runners. But in terms of infrastructure, both MacStadium and Circle CI generously donate MacOS resources to GIMP (MacStadium has provided a Mac Mini and Circle CI the full CI experience including runners). I believe one of these approaches are likely to be more long-term effective for GLib and GTK.
I don’t share this belief. Unless GIMP is testing against GLib’s and GTK’s main branch, then CI testing isn’t going to catch build issues and regressions in merge requests.
Since this topic has been linked in a couple of places, and I’ve seen a lot of misunderstanding, I want to clarify a specific issue: this topic is about the GitLab CI pipeline running on macOS. We are asking for help in properly maintaining the macOS machine we use for running the CI pipelines of GLib and GTK.
Of course, the retirement of the macOS CI pipeline will have an effect on the ability to avoid build issues and regressions on macOS; but right now we’re pretty focused on ensuring that we can reliably build GLib and GTK merge requests.
Any offer of help with macOS support is, of course, more than welcome, but that is independent from the issue at hand.
I think you misunderstood @lukaso’s suggestion. What they’re talking about is following GIMP’s example and applying one of its approaches to GLib/GTK’s CI (i.e. contacting MacStadium or Circle CI for free resources) rather than relying on GIMP’s builders to catch regressions in GTK.
FWIW, both QEMU and libvirt use Cirrus CI free service for testing macOS builds. It is possible to integrate this into a GitLab CI pipeline dashboard, so you still have a single source of truth for CI results, by using the ‘cirrus-run’ program (GitHub - sio/cirrus-run: Command line tool to execute jobs in Cirrus CI).
What happens is that a job in GitLab CI runs (on a Linux shared runner in gitlab.com context), and invokes ‘cirrus-run’. This tool talks to the Cirrus CI REST API to spawn a CI job on macOS (aarch64) hardware. That job pulls the code to be tested over from gitlab and runs the macOS build. ‘cirrus-run’ downloads the resulting job logfile from Cirrus CI and spits it out on the console. Thus to all intents it appears as if you have native macOS support in your GitLab CI system.
The main caveat here is that while Cirrus CI does not impose total monthly running time limits on free usage, it does have a concurrency limit on the number of jobs you get to run in parallel. In the case of their macOS platform this limits you to 1 job, ie everything serialized. I don’t know what kind of load GNOME has to cope with in terms of number of macOS pipelines running in upstream repo context, per day and running time per pipeline ? You can potentially pay for extra compute credits to raise the concurrency limit.
The second caveat is that you need to register a Cirrus CI API token against any repo that needs to run cirrus-run. IOW, if your contributor’s pipelines are running in the context of their fork repo, not upstream repo, they won’t get macOS coverage unless they’ve setup the API token in their fork too.
It looks like those CI jobs run outside of GitLab. Do you know how results are made available in MRs for GIMP? Or is it more of a weekly schedule thing, with failures automatically reported as issues against GIMP? I’ve tried to find some output from the system but haven’t found anything (but I’m probably just rubbish at searching).
The main caveat here is that while Cirrus CI does not impose total monthly running time limits on free usage, it does have a concurrency limit on the number of jobs you get to run in parallel. In the case of their macOS platform this limits you to 1 job, ie everything serialized. I don’t know what kind of load GNOME has to cope with in terms of number of macOS pipelines running in upstream repo context, per day and running time per pipeline ? You can potentially pay for extra compute credits to raise the concurrency limit.
The Cirrus service sounds workable. It would be better if we could tie the runner straight into the GitLab runner controller, since that means less manual glue to maintain in GLib. In terms of resource requirements, GLib should be fine with anything from testing every MR on macOS, through to running a single macOS build against main once a week and automatically filing an issue if the build/tests fail.
It would be OK for macOS to only run against origin and not against forks (that’s actually what we already had, due to the old macOS runner not being virtualised).
