Applications that need data files in the build directory

Applications
1

My application (during a non-offline build) downloads one or more data files that are used as inputs to code generation.

The CMake logic looks something like this:

set(CACHE_DIR "${PROJECT_BINARY_DIR}/../caches CACHE PATH "Location of shared cache of downloaded files")

set(COMPONENT_CACHE "${CACHE_DIR}/ComponentName" CACHE PATH "Location for Component's downloaded data")

if (ENABLE_NETWORK_DOWNLOADS)
   my_file_download_helper_func("some url" "${COMPONENT_CACHE}/some_file.dat")
else()
   message(STATUS "Skipping download of some url, expecting it to be at ${COMPONENT_CACHE}/some_file.dat")
endif()
my_custom_command_helper("my_code_generator" "${COMPONENT_CACHE}" "Component.h" "Component.cpp")

When translating to a flatpak manifest for offline builds, I handle this with two steps:

However, when building the project via Builder (i.e. opening the folder with my local flatpak manifest that has these two source entries + a “dir” file source as the final module), The two sources for downloading the file and for running the shell script are not run when building the project.

It looks like so:

      "sources": [
        {
          "type": "file",
          "url": "https://raw.githubusercontent.com/publicsuffix/list/32c68ce9d52e12df5b914b2b248cb893147301f7/public_suffix_list.dat",
          "dest": "Caches/PublicSuffix/",
          "sha256": "e79e372bcc6fcdb51f7a31e3c0c504530838432669af2ac544d2491de0a86030"
        },
        {
          "type": "shell",
          "commands": [
            "# FIXME Is there an easier way to set an absolute path in config-opts?",
            "echo set\\(LADYBIRD_CACHE_DIR \"$PWD/Caches\" CACHE STRING \\\"\\\"\\) > Cache.cmake"
          ]
        },
        {
          "type": "dir",
          "path": "../../../",
          "skip": [
            "Build",
            "Tests/LibWeb/WPT"
          ]
        }
      ],

After configuring and hitting the build button, neither of those two files can be found via a find ~/Projects/.gnome-builder.

Multiple sources for the dependency modules works just fine.

Is this a known issue? Is there another approach I should use? This works just fine with the flatpak-builder CLI tool.

2

Hi,

IIRC, Builder will ignore the last element of the “modules” array of the flatpak manifest, and assume it’s the current folder and everything is already available there.

Is your data file required during the build phase, or is it just to be installed later?
In the 2nd case, just add another “modules” element to manage that data file.
In the 1st case, you will have to fetch it manually in your sources folders. If you want to test the whole offline flatpak build, then use the flatpak-builder tool instead of Gnome-Builder.

3

The extra files are required at build time of the main project (i.e. the final sources module).

I don’t mind having an online build for the final sources module inside Builder, as I’ve got flatpak-builder for the offline build as you say. However, my build fails inside builder at the moment because the at-sources-time generated Cache.cmake doesn’t exist in the Builder case.

Unless there’s a way to pass the build-time path to CMake that I need via the config-options array directly, I’m stuck having to use two separate flatpak manifests, one for flatpak-builder and one for Builder, right?

That is, I need to pass -DMY_CACHE_DIR=/path/to/where/sources/are/dropped. As far as I can tell, CMake configure args in the manifest aren’t evaluated in any way. So I need a CMake script to evaluate some expression. In my case I choose $PWD, though $FLATPAK_BUILDDIR or so could work as well.

Seeing as I have like twenty sources modules for dependencies that are not in the KDE SDK, duplicating that file is not very appealing.

4

I’m not sure how to manage that properly… I don’t think a separate manifest would help.

I suggest that way: in your CMake files, add a check if that additional file is present (usecase: offline build), and if not then download it like you did before (usecase: Gnome-Builder).

5

This topic was automatically closed 45 days after the last reply. New replies are no longer allowed.