Mocking framework for GLib

Development
1

Hi everyone,

I’ve been thinking about the current state of unit testing within the GLib ecosystem, specifically regarding how we handle mocking, as of now, there is no standardized way to mock non-GObject modules.

Mocking in GObject

Currently in GObject, mocking can be succesfully implemented by deriving the object class and overriding the virtual methods (provided the class has them) with a mocked version. However, when you need to test a module that depends on more complex functions—such as kernel syscalls, user-driven functionality, or non-GObject C functions—this becomes significantly harder to test.

Mocking non-GObject functions

Currently, mocking non-GObject C functions often requires messy linker tricks (like -Wl,--wrap) just to make mocks to work. This lack of a standardized, ergonomic mocking framework for GLib often leads to developers skipping deep integration tests or maintaining brittle.

Solution

I’m proposing to develop a mocking framework for GLib, which would let the developers to mock non-GObject functions, it will be implemented by using GCC’s wrappers and MSVC’s weak linkage, following GLib philosophy that it must be cross-compatible between all compilers and platforms.

References:

GLib Mock (glib-mock), Mocking framework, MVP stage
2

The API abstraction I’m thinking is the following: A single (1) G_DEFINE_MOCK macro for defining a mock.

#if defined(__GNUC__)
# define G_DEFINE_MOCK(return_type, func_name, ...)    \
    extern return_type __real_##func_name (__VA_ARGS__); \
    return_type __wrap_##func_name (__VA_ARGS__)
#elif ... /* Other compilers and platforms */
#else
# error "Cannot do mocks"
#endif

Example of usage:

/* Mocking read syscall */

static struct {
  gboolean must_fail;

  /* Other stuff... */

} mock_state;

extern int errno;

G_DEFINE_MOCK (ssize_t, read, int fd, void *buf, size_t count)
{
  if (mock_state.must_fail)
    {
      errno = EIO;
      return -1;
    }

  ssize_t n_read = MIN (sizeof ("Hello World!"), count);
  memcpy (buf, "Hello World!", n_read);
  return n_read;
}

void test_read (void)
{
  mock_state.must_fail = FALSE;
  /* Here you would really call the module function you are testing,
   * that finally calls into read syscall */

  /* But, for demonstation purposes, I'm just going to call read */
  char buf[4096];
  ssize_t n_read = read (0x1234, buf, sizeof (buf));

  g_assert_true (n_read > 0);

  mock_state.must_fail = TRUE;

  n_read = read (0x1234, buf, sizeof (buf));

  g_assert_true (n_read == -1);
  g_assert_true (errno != 0);
}

int main (int argc, char **argv)
{
  /* Add GLib test boilerplate */

  return g_test_run ();
}

It can also be abstracted into two macros, one for declaring a mock (G_DECLARE_MOCK) and the other for defining it (G_DEFINE_MOCK), following the same boilerplate pattern used by G_DECLARE_TYPE and G_DEFINE_TYPE. This two macros approach may be more stable across multiple compilers.

3

As to why I want to develop this framework:

I wanted to create unit tests for the relatively new GNotificationBackend for Windows, but I got stopped by the fact that the backend calls into WinAPI functions, and some features that I want to test requires user interaction explicitly.

4

There doesn’t actually seem to be anything here which needs to be in GLib, so why don’t you put it into a new library and try it out?

If it turns out to be super useful then it could potentially be moved into GLib later. But putting things in GLib first means they’re subject to GLib’s strict API stability guarantees, which is not good for prototyping.

2 Likes
5

It is indeed in prototype phase, creating a library is a great idea. I think I can use the GNOME’s GitLab to host the repository right?

6

As long as the license is OSI-approved, you can use your personal namespace on the GNOME GitLab instance.

1 Like
7

I think my last message must have raced with yours.

For creating unit tests for the Windows GNotificationBackend, the mocking code has to live inside GLib. It can be private though.

In cases like this, I suspect it would be clearer to create specialised mock code for the handful of APIs you need, and not try to generalise it. My intuition is that generalising it would make it harder to understand. But YMMV.

8

For creating unit tests for the Windows GNotificationBackend, the mocking code has to live inside GLib.

You mean in the gio/tests directory or just in gio?

I wanted to follow a “linker-based” approach because I thought it would be the less invasive with the original source code, as well as having the advantage that the mocking code can live in the tests directory.

In order to enable the current source code for the Windows GNotificationBackend to be mocked, it would need to create the mocked versions conditionally with a hypothetical #if defined(IN_TEST) \ #define ... macro, which I think would be anti-climatic to have in the source code rather than in test files.

9

Ahh, but that’s what abstractions solve, make it easier to understand.

I started development of this prototype, it consist of a header file with 2 macros and a (for now empty) python script that when run will instruct the dynamic linker to change the original implementation with the mocked one provided through arguments as an .so, the script then will add those to the variable LD_PRELOAD/DYLD_INSERT_LIBRARIES (linux/macos) and will run the test executable with that mocked version. For Windows, currently is unimplemented but it will perform what is known as IAT hooking.

I mentioned early that was going to use GCC’s wrappers, but it isn’t portable, and MSVC’s weak linkage requires changes to the original source code.

References:

Windows:

Linux:

Macos:

10

In the gio/tests directory.

That would potentially work in this case, as you can tailor it to the MSVC linker. In the general case, a linker based approach is very hard to get working across multiple platforms because they all differ in how they arrange linkage.

See, for example, the socket-listener test in GIO, which interposes in loader symbol resolution to override some syscalls. It only works on Linux, though, and would need DYLD_INTERPOSEsupport to be added for macOS. I don’t think anyone’s tried to get it working on Windows.

So yes, it’s less invasive in the code under test (which is important), but it’s hard to get right across multiple platforms. I’d stress that point actually, for the benefit of everyone else: if the code under test has branches which are only used when testing (e.g. something like if (enable_mocks) call_mock() else call_real_function()) then you’re not really testing it at all. So a goal would be to not require any changes to the code under test to make it testable. Sometimes that’s unavoidable, and in those cases it’s preferable to introduce changes which don’t branch, like providing callbacks or an interface (e.g. which are called unconditionally) to abstract the syscalls.

See also glib/tests/getpwuid-preload.c · main · GNOME / GLib · GitLab (and the loading code: glib/tests/meson.build · main · GNOME / GLib · GitLab ), which takes a different approach (basically being loaded via LD_PRELOAD, but of course that differs between platforms) which is equally as complicated.

Yes and no, I think it depends on the situation. If the abstraction changes the form of the code radically from how it would look if it were just open-coded, and makes the form more complex, that can make things overall harder to understand.

Sometimes a helper function (which your test code calls out to, to abstract platform differences) can be clearer than a wrapper abstraction (which wraps all your test code and tries to do everything). I would also suggest it might be simpler to explicitly focus on syscalls (if you aren’t already), rather than arbitrary function calls, as those are already explicitly an API boundary which is quite well defined and amenable to being overridden (i.e. the addresses of syscalls always have to get loaded at runtime).

Ultimately the right shape for test harness code like this can probably only be found by trying something and seeing how it turns out. I’m interested to see how it turns out :grin:

11

Indeed, but in the end a syscall is just a function (in glibc, WinAPI, etc.) What this means for the mocking prototype is that I must add a documentation comment saying that the function added must be a dynamically loaded function in order for it to work then, preferably a syscall function, due to the fact that it uses dlsym to get the original function (in linux and macos).

12

When the mocking framework reaches some MVP stage, I will create a new post here, see ya.

1 Like
13

Wait, I’m over-engineering​:rofl:!

Regarding using LD_PRELOAD (did just test it on Linux), you can just define the mocked syscall in the test main file and ldd will succesfully load that by default instead of the libc one for all shared objects, you can retrieve the original implementation with dlsym(RTLD_NEXT). For macos, I don’t have a VM to test for it, but the DYLD_INTERPOSITION macro should work like that as well. For Windows, you still have to perform IAT hooking on runtime.

All of that can be implemented in pure C code without the help of a Python script.

Ultimately, no LD_PRELOAD and DYLD_INSERT_LIBRARIES was needed, but still there may be cases and cases :grin:.