Hi, Milan - first of all, thanks for maintaining Evolution and its related modules for all this time. I use it every day and I appreciate the amount of work that it takes to keep it running.
Setting up CI is indeed overwhelming, but don’t despair!
Would you like some help in setting up CI for Evolution and its modules? I’ve done this a few times, for librsvg, gnome-crosswords, libipuz, at-spi2-core, and more or less know my way around things. I think Evolution and e-d-s are the kind of large, old modules that could greatly benefit from some of the toys that can be plugged into a CI pipeline:
generate a test coverage report (librsvg and libipuz use grcov for their reports).
run various sanitizers (e.g. the asan-build and static-scan jobs in libipuz).
The biggest chunk of work is setting up a reproducible build environment, but the Freedesktop CI templates make this quite convenient. I can help with that if you walk me through the build process; I see that Evo has docs about this already. We can also automate the release process as much a possible so you only have to write the NEWS, basically, and do much fewer manual steps.
In short: I can dedicate some time to setting up CI for Evolution, if you’d like, but I’d need your help with dealing with cmake and following the proper procedure to set up the build environment. Would you like this kind of help?
I’m on vacation through the 48.alpha deadline so I won’t have time to updated any of my modules given this extremely late notice. I’ll need release team to handle the following modules:
gnome-text-editor
gtksourceview
jsonrpc-glib (might be able to skip)
template-glib (might be able to skip)
gnome-builder
libspelling
libdex
d-spy
gom
libpeas
libpanel
sysprof
though I also need to do release for
chergert/ptyxis
chergert/manuals
though they are not GNOME modules yet.
Alternatively, maybe giving us more than one month notice during the peak holiday time would be the better option.
While recommended (to prevent issues during the beta or rc releases), it is not necessary to provide an alpha release.
Note that at the moment only repositories in the GNOME and Incubator namespaces can use the release service. Perhaps you can open an issue if this affects modules that were previously using d.g.o for their tarballs.
As an update, if you are using the Flatpak CI template, it now always runs meson dist and thus you can use the new release service for your app with minimal effort.
Yes, except if your project previously released on download.gnome.org, then you may file a ticket to request an exception to allow use of the release service.
there is no CI_COMMIT_TAG defined on usual commit - in such case the tarball should not be created, from my point of view, but the example .gitlab-ci.yml seems to create it always, or at least my “adaptation” of it does;
the example contains stage release, but no stage: release section; that’s confusing;
the evo & co. projects use the Flatpak CI; the tarballs are created with git archive ... command, but there is no git in the Flatpak CI. Could it be added, please? Or do I need to carry on some Debian/Fedora/whatever image where the git will be available, update it regularly to stay up-to-date just for the git command?
My idea is to create the tarball and add it into the artifacts only if needed (like when the CI_COMMIT_TAG is defined) and create only the tarball itself, because other tests are done in other stages.
Also, when the pipeline fails for whatever reason, will restarting it contain the CI_COMMIT_TAG or not? It should, if the former (failed) job/pipeline did. Maybe it does, I do not know. I only know that the pipeline can fail for various reasons, including infrastructure problems.
The CI_COMMIT_TAG variable is present only if the pipeline was triggered by a tag creation. This opens the option to add a rule to your job to make it run only if the pipeline was triggered by the tag. Example:
# ... other stuff in gitlab ...
your-job:
# ... job stuff ...
rules:
- if: $CI_COMMIT_TAG # && $CI_COMMIT_REF_PROTECTED # <-- uncomment to run only on a tag that is also protected
Yes, restarting a pipeline/job should trigger it in the exact same state with the same env vars it was originally created with, including the CI_COMMIT_TAG if it originally existed.
added a section about how to adjust the CI for projects using Flatpak
CI (scroll a bit under the minimal example).
Hi,
this is CMake, not meson, there is no ‘meson-dist’ directory under the
flatpak build directory here. Back at the square 1, I need the git
command here. I noticed the ‘tarball’ stage runs with
I tried to switch to the “quay.io/gnome_infrastructure/gnome-runtime-
images:gnome-master”, which the flatpak stage uses, and it does have
the git command, thus it’s a bit better. It still fails, just with a
different error:
$ git archive --prefix=${TARBALL_PATH_PREFIX}/ HEAD | xz -z > ${TARBALL_NAME}
fatal: detected dubious ownership in repository at '/builds/mcrha/evolution-data-server'
To add an exception for this directory, call:
git config --global --add safe.directory /builds/mcrha/evolution-data-server
Looking what it is, the files are extracted as root:root, with one
build:build file:
$ ls -la
total 8928
drwxrwxrwx. 1 root root 548 Dec 27 09:14 .
drwxrwxrwx. 1 root root 92 Dec 27 09:14 ..
-rw-rw-rw-. 1 root root 210 Dec 27 09:14 .dir-locals.el
-rw-rw-rw-. 1 root root 195 Dec 27 09:14 .editorconfig
drwxrwxrwx. 1 root root 10 Dec 27 09:14 .flatpak-builder
drwxrwxrwx. 1 root root 86 Dec 27 09:14 .git
-rw-rw-rw-. 1 root root 33 Dec 27 09:14 .gitignore
-rw-rw-rw-. 1 root root 3060 Dec 27 09:14 .gitlab-ci.yml
-rw-rw-rw-. 1 root root 0 Dec 27 09:14 AUTHORS
-rw-rw-rw-. 1 root root 36621 Dec 27 09:14 CMakeLists.txt
-rw-rw-rw-. 1 root root 25290 Dec 27 09:14 COPYING
-rw-rw-rw-. 1 root root 366 Dec 27 09:14 ChangeLog
-rw-rw-rw-. 1 root root 0 Dec 27 09:14 HACKING
-rw-rw-rw-. 1 root root 30 Dec 27 09:14 MAINTAINERS
-rw-rw-rw-. 1 root root 479821 Dec 27 09:14 NEWS
-rw-rw-rw-. 1 root root 655 Dec 27 09:14 README
drwxrwxrwx. 1 root root 136 Dec 27 09:14 cmake
-rw-rw-rw-. 1 root root 6631 Dec 27 09:14 config.h.in
drwxrwxrwx. 1 root root 3990 Dec 27 09:14 data
drwxrwxrwx. 1 root root 540 Dec 27 09:14 docs
-rw-r--r--. 1 build build 8537944 Dec 27 09:06 evolution-data-server-nightly.flatpak
-rw-rw-rw-. 1 root root 1112 Dec 27 09:14 evolution-data-server.doap
-rw-rw-rw-. 1 root root 655 Dec 27 09:14 evolution-data-server.pc.in
-rw-rw-rw-. 1 root root 5910 Dec 27 09:14 iconv-detect.c
drwxrwxrwx. 1 root root 1134 Dec 27 09:14 po
drwxrwxrwx. 1 root root 236 Dec 27 09:14 src
drwxrwxrwx. 1 root root 232 Dec 27 09:14 tests
Following the hint from the git command failure (see above), it makes
the tarball. This tarball seems to be a correct tarball.
One last thing, is there an easy way to verify the ${CI_COMMIT_TAG}
contains an expected data, aka in this case:
? Creating the tarball for other tags is useless.
By the way, may not be the tarballs marked with anything like
expire_in: 14 days
in the CI? There is no need to keep them ad infinity in the CI, I
guess.
I also verified a failed ‘tarball’ job with a tag is restarted with the
proper CI_COMMIT_TAG variable set. That’s fine.
The Release notes seem to use the Tag commit message.
I’m not going to copy the NEWS file changes into the tag, not talking
the “Tag 1.2.3 release” is useless for the release notes. Maybe it’s
only for the sandbox though.
Hi,
so, here it goes. I just realized evolution-ews’ gnome-47 branch is
using wrong branch for the build. It creates proper tarball (it seems),
but the Flatpak build runs with a wrong branch (master), not
the gnome-47, for the dependencies.
The failed build is here:
The Evolution changed API, which is triggered by the build failure. I
do not know how this works, I guess the jq commands in
the .gitlag-ci.yml of the evolution-ews somehow modify the Flatpak
manifest file for the evo-ews itself, but not for the dependencies. I
can fix it easily by manually changing branch: master to branch: gnome-47 in the manifest, but the tag 3.54.3 already exists
and points to a correct code, but to a wrong CI script.
What now?
You know, I’m a human, I do mistakes, a lot of mistakes. If the only
answer is “do a 3.54.4 release”, then I’m against it, also because it
has consequences on the dependencies or the build scripts (and the
package build scripts).
I cannot delete the 3.54.3 tag in the web UI, even the error message
is empty (yeah, empty, no reason, just almost empty red bar, only with
a red exclamation mark in a circle on the left and an ‘x’ on the
right). There had been mentioned somewhere that the tags cannot be
deleted, but frankly, if one cannot correct mistakes, then what are the
machines for?
Hi,
I corrected the CI, two commits, not one, of course. I also noticed the
build from the tag does not contain proper branch name, thus no way to
update the Flatpak manifest on the fly for correct dependencies.
The fixed CI is here:
I need to delete the 3.54.3 tag and create a new one, from the last
commit. The GitLab web UI offers to delete the protected tags, thus
it’s possible, but the GNOME’s GitLab instance fails to do it (see the
previous message).
Tags are permanent and can never be deleted. Best practice is to just release a 3.54.4. In the past we have used tags like “3.54.3-really” but this is surely incompatible with the new release process.
I suppose you mean “in the GNOME world”, because the GitLab web UI says
something else. Why, if I can ask? The decision does not make sense to
me. (I’m not sure, I might be said that earlier, but it’s too long ago,
I already forgot it.)
Best practice is to just release a 3.54.4. In the past we have used
tags like “3.54.3-really” but this is surely incompatible with the
new release process.
Doing so has consequences involving package maintainers. I prefer to
avoid breaking the things in a stable series. In short,
the evolution-ews X has a dependency on the evolution-data-server and
evolution of the same X version. There is no 3.54.4 release of these
two, which means, if going by your ugly workaround path:
a) change the CMakeList.txt to not require 3.54.4 in the build time
b) all the package maintainers to modify their package building scripts
and package dependencies similarly
c) revert the changes from the a) for the next stable release
d) revert the changes from the b) by all the package maintainers for
the next release.
That’s a lot of work involving out-of-GNOME-world people, which may
cost time. For what? The price feels too high to me.
Doing 3.54.3.1 would be very similar problem. Not mentioning there will
be absolutely no code change, the new version number would be wrong and
confusing. All of this complication just because someone is stubborn
and does not understand people do mistakes? I do mistakes.
A proper workaround would be to release with the old good ftpadmin,
if you insist the tags in the GNOME world cannot be deleted. I’ll be
happy to provide the tarball, in case the infra/release folks can run
the command on some machine (no need to enable it just for this
project).
Because git is distributed, and deleting the tag in one repo does not delete it in everybody else’s copy of the repo. It gets really confusing when your 3.54.3 tag is different from other developers’ tags, so GNOME simply prohibits this.
I think ideally your CMake build scripts ought to be able to handle a 3.54.3.1 release. There are many valid reasons why you may want to do a quick release without bumping the minor version that carries special meaning for the evolution projects.
Some distros use the git tag as part of their packaging and may actually package the new release very quickly.