Releasing GNOME modules

Community
,
1

Never send a human to do a machine’s job.
– Agent Smith, The Matrix (1999)

For a long while, now, maintainers of GNOME modules have been following a pretty standard process to releasing new versions of their projects:

  1. build a release archive locally
  2. copy the release archive to a GNOME server
  3. SSH into the server and call a script

This process is manual, error prone, and requires giving SSH access to a server somewhere, opening up the possibility of misuse.

Well, not any more.

Maintainers can now rest easy, and use the GNOME release service written by Stefan Peknik which integrates with an existing GitLab CI pipeline. All you need to do is to set up a CI job that generates the release archive and stores it as an artefact, and use the CI template that will take care of uploading the archive to the right place, and notify all the various services in the GNOME infrastructure.

The new service replaces the SSH access to master.gnome.org and the ftpadmin script, which will be retired by the end of the week.

The GNOME handbook will soon be updated with instructions and examples for maintainers.

If your project does not have a CI pipeline, you can ask for help in the release team room on Matrix.

Make sure to update your project before the 48.alpha release deadline of January 4, 2025!

5 Likes
2

Hmm, looking at the CI template, looks like it builds against alpine and not gnome SDK. How do we ensure it has new enough deps? For some modules you may have to build half of GNOME, like for libadwaita I’ll almost invariably need recent GTK

Another thing is that specifically libadwaita releases must be configured with -Dgtk_doc=true and then you have to run ninja, or otherwise a dist hook will tell you to do that - so that tarballs have pre-built docs

So I suppose the template is meant to be used as a draft and not included as is?

3

You can use the template as the basis, and provide your own script and image directives:

release-build:
  stage: build
  extends: .meson-dist
  # This will take precendence over the `meson-dist` template
  image: fedora:41
  # This will run before `meson setup` and `meson dist` from the template
  before_script:
    - sudo dnf install -y ${YOUR_DEPENDENCIES}
    - some_setup_script.sh
  rules:
    - if: $CI_COMMIT_TAG

If you have specific needs, you can open code your own dist CI job; the only requirement is having the release archive stored as a pipeline artefact.

Additionally, we can provide other templates, just like we do for publishing Flatpaks to the nightly repository, or include additional options in the existing template.

2 Likes
4

Makes sense. Another thing I wonder is how to test if the pipeline works during the initial setup? As I’d rather test it before and not after tagging the next release

1 Like
5

You should create a separate CI job that runs meson dist on all branches, but not on merge requests; this way your project is always in a releasable state, and the “release on a tag” job is merely a formality.

1 Like
6

The part that makes me anxious is getting the path TARBALL_ARTIFACT_PATH for the tarbal right in the release-build

7

The path is always in the form of:

${MESON_BUILDDIR}/meson-dist/${CI_PROJECT_NAME}-${CI_COMMIT_TAG}

assuming:

  • your project name and the name of the repository are the same, which should always be the case
  • the tag is the version number in your project, which it really should be unless you enjoy downstream distributors being in pain

That’s why it can be placed in a template.

1 Like
8

The libxml2 and libxslt release tags currently have a v prefix unlike the filename of the tarball. Unfortunately, I can’t find a way to strip the prefix from $CI_COMMIT_TAG. Does anyone have an idea?

9

To save early adopters some pain, here is what I learned in an afternoon of trying to get this to work:

If you want to run meson dist in gitlab ci, you need to have

git config --global --add safe.directory $CI_PROJECT_DIR

before the meson dist command in your script. Otherwise, you get a cryptic error message from meson:

Dist currently only works with Git or Mercurial repos
4 Likes
10

Here is an example of a working setup:

A few things to note:

  • We decided to just use one of the regular jobs to produce the tarball
  • Note git config --add that I’ve mentioned before
  • You need to make sure that the build job producing the tarball and the dist one both run on tags
  • Tags need to be ‘protected’ for the release template to work (We are considering making that the default setting for all tags in the GNOME group)
2 Likes
11

Hi,
my projectS are never in the release state. Part of the release are
some manual steps, like updating the NEWS file and the metainfo.xml
files for the apps. I never push the changes to the git before I verify
these things work as they should.

The new steps, which will be in the handbook soon, as you said, I guess
will be along:

  • locally update the NEWS file
  • test the release (make disttest and the like)
  • push the NEWS update to the git
  • open a web page and click on some button
  • enjoy your tea and newspapers read until the CI does what you already have done locally; hopefully in some not distant future
  • switch back to the local command line
  • tag the release with the proper (possibly signed) tag and name relevant to your project
  • do post-release version bump
  • possibly add the release info to the GitLab Releases page (that’s only for one project I work with)

You know, I’m a human, I do mistakes, in fact, I do a lot of mistakes.
The manual work needs to be verified before pushed to the git. The
duplicated work and the waiting for something I’ve already locally done
doesn’t sound like a good step forward. I doubt you can avoid the NEWS
updates, that needs human intervention.

How do I test the script does what it should do without actually doing
the release someone might want to use? If “clicking the button on the
web site” means also uploading to the download.gnome.org and notifying
by mail and all of that, then it’s nothing for testing, right? This is
not meson, thus your template (for meson) simply doesn’t work. Writing
something from scratch is a problem (remember, I do mistakes). I also
do not want some random person which passes around (accidentally or
not) pressing the button in the web UI. That should be done on purpose,
by the maintainers only; aka a controlled environment. After all, the
maintainers know the details of their projects.

I agree with Alice, the dependencies of the evolution-data-server,
evolution, evolution-ews and evolution-mapi are very tight, they
require exactly the same version as they are (or newer). That can be
tricky to achieve.

Speaking of the evolution-mapi, it requires OpenChange, which requires
samba. It does not have any real “make disttest”, but the “make dist”
is available only after the project is configured, thus the CI would
take ages to build the dependencies.

The evolution-etesync requires libetebase, which is Rust, which may
require Internet access. The libetebase is not part of the GNOME, it’s
hosted elsewhere.

I guess the dependencies will need some special CI/Docker image with
the dependencies preinstalled? Docker is not my friend, and updating
the images every few months is also something far behind my maintainer
abilities, at least yet.

Maybe a way to workaround the pain with those complicated cases would
be to rewrite the make dist to a bash script? It may surely speed up
the tarball build time significantly and considering all the runtime
tests are finished locally anyway.

I understand that this is a step forward and works for simple cases,
but the life is not simple. I guess.

Bye,
Milan
12

Being in a releasable state does not mean “being released”. A releasable state means that everything outside of release-specific bits, like adding release notes, is already in the repository, and that the dist process is merely a formality.

Missed this bit.

Part of the process depends on protecting the tags in your repository, so that only people with Maintainer access can push them.

13

Excuse me, I’m new to GNOME Discourse, what are “GNOME modules”?

14

“GNOME modules” are projects that are part of the GNOME release, hosted under the GNOME and Incubator groups in GitLab, and that have release archives deployed to download.gnome.org.

1 Like
15

I’d prefer to have a protected tag namespace only for releases, like rel/* for example. But this will require to strip a prefix from the tag as well. As mentioned above, I haven’t found a simple solution to do that.

16

We should probably modify the CI template to add a prefix variable.

Or alternatively, you can copy the template and experiment with it to fit your needs.

17

I don’t know the specifics of your project, but in general, everything you’ve talked about like versioned dependencies and external dependencies downloaded from the internet has been done in CI by many projects. It seems most of the trouble you’re describing is just a normal setup of a CI pipeline that is unrelated to release management.

18

A protected namespace already exists for releases, it’s git’s annotated/signed tags (as opposed to lightweight tags).

19

If this is with bash, or if bash can be used, the solution is ${CI_COMMIT_TAG#v}

It isn’t standard shell, see in the bash manual:

${parameter#word}
${parameter##word}

20

This is with GitLab’s internal variable expansion mechanism which doesn’t have any of these features.