Libnice should have libsoup as depency?

We are still struggling with libsoup 2 and 3 issues.

And I wonder why for libnice no libsoup dependency is reported:

head ~/.nimble/pkgs/gintro-#head/gintro/nice.nim 
# dependencies:
# GObject-2.0
# Gio-2.0
# GLib-2.0
# immediate dependencies:
# Gio-2.0
# GObject-2.0
# libraries:
# 
ldd /usr/lib64/libnice.so
libsoup-2.4.so.1 => /usr/lib64/libsoup-2.4.so.1 (0x00007ff2f32a5000)

So ldd reports the libsoup dependency, but https://gnome.pages.gitlab.gnome.org/gobject-introspection/girepository/GIRepository.html#g-irepository-get-dependencies ignores it.

For webkitgtk that libsoup dependency is reported fine.

When libsoup dependency is not reported at all for libnice, then it is impossible to avoid the version 2/3 conflict.

Its not a direct dependency. The direct dependency that pulls in is gupnp-idg, via gupnp. And IIRC its not used in its public API eiterh.

1 Like

But unfortunately gobject-introspection can crash, when libnice is used together with a not matching libsoup version. So I wonder, what may happen when libnice and libsoup are used in an app together from Python or JavaScript.

But for my Nim bindings, and the problem that gobject-introspection can crash when multiple versions of libsoup are loaded within the same process, I just had an idea: I can use g_irepository_get_loaded_namespaces() to check which namespaces are already loaded (automatically), and I think there is also a function to get the version numbers. Then I should be able to avoid version conflicts.

I don’t think this will work. GIR cannot know about that. GUPnP is a private implementation detail of GUPnP-IGD which is a private implementation detail of NICE. the GIR of NICE doesn’t even come near to libsoup.

You are right. g_irepository_get_loaded_namespaces() does not report libsoup after libnice is processed by gobject-introspection. But still gobject-introspection somehow loads libsoup, which may generate crashes when symbols from libsoup3 and libsoup2 are loaded in the same process. This is really an ugly problem, which some ArchLinux users have since a few months. My Gentoo-Linux does not have libsoup3 already, so it is difficult to reproduce for me. And now more and more distros ship libsoup3, so more people will get problems. Fully ignoring libnice may help, that is not generating bindings for libnice at all. But we had a user who used libnice.

Actually I can not reproduce the ArchLinux issues from Cant install gintro on Arch. Using libsoup2 and libsoup3 in the same process is not supported. · Issue #190 · StefanSalewski/gintro · GitHub

libsoup3 symbols detected. Using libsoup2 and libsoup3 in the same process is not supported.

I am using Gnome as provided by recent Gentoo Linux, and just installed latest libsoup 3 from git sources. And install of the Nim bindings works fine, and I get additional the soup3.nim file.

I have to admit that I was able to reproduce the issue at the end of last year, but seems to be fixed. And indeed, we got issue reports only from ArchLinux users, like [Sugestion] remove libsoup · Issue #193 · StefanSalewski/gintro · GitHub. So I will hope that all is fine now, and will close this thread.

libsoup2 vs libsoup3 is really a build/deployment/distribution problem and nothing you specifically should have to handle. Any application must only use a single version of libsoup directly or indirectly, and it’s the job of whoever built the application and distributed its set of libraries to ensure that.

1 Like

nothing you specifically should have to handle.

Yes, that was my guess and hope too. But it is a fact that some of the ArchLinux users got these

libsoup3 symbols detected. Using libsoup2 and libsoup3 in the same process is not supported.

issue, when the package install script runs gobject-introspection to create the Nim bindings. The issues started when ArchLinux got the libsoup3. For a short period I was able to reproduce it, but only with a strange mix of regular Gentoo packages and some libs like libsoup compiled from git sources, which is generally a bad combination. As all is fine now for me, and likely for all non ArchLinux users, I assume an error in Arch packages or maybe gobject-introspection. I looked at the gobject-introspection issue tracker this morning – some people including Mr. Bassi seems to be still working on gobject-introspection, so maybe they have fixed it somehow. Maybe gobject-introspection was loading libsoup when processing libnice for some reason, which is definitely not a good idea. This type of error would definitely occur, when in one process gtk3 and gtk4 is loaded at the same time, or when libsoup 2 and 3 are loaded at the same time. But my Nim bindings generator runs two fully distinct processes, so all should be fine. For all the packages that we currently try to install for Nim, the webkitgtk is the only one, which reports a libsoup dependency. And for that, we use two separate processes – one with webkitgtk compiled for gtk3 with libsoup2, and one with webkitgtk compiled for gtk4 with libsoup3. But from the ArchLinux users error reports the problem was more when generation libnice bindings, which do not report a libsoup dependency, and so should never cause a libsoup version conflict during bindings generation. So finally, I really hope that all is fine now.

Actually Gentoo and LinuxFromScratch still avoid installing libsoup3 still, see

libsoup-2.74.2

net-libs/libsoup – Gentoo Packages

That reason is the dynamic loader ld.so. You cannot avoid that.

And my 2ct on that: Gentoo and Arch users that cannot handle this kind of issues themselves might consider to re-evaluate their choice of Linux distribution.

3 Likes

That was a very good hint. Well, I was thinking about a relation with ldopen() myself already, I was even going to temporary delete some libs to see if the bindings generation process would complain. But after your post, I just asked Google about how to find opened libs, which gives libraries - How to see the currently loaded shared objects in Linux? - Super User

The interesting point is, that when my bindings generator runs generation libnice bindings, I get no output for this command

grep libnice.so /proc/*/maps

with my default libnice from Gentoo. But when I install latest libnice from git to /opt, I get this

grep libnice.so /proc/*/maps
/proc/20888/maps:7fcd77e27000-7fcd77e31000 r--p 00000000 00:0e 23763188                   /opt/gtk/lib64/libnice.so.10.11.0
/proc/20888/maps:7fcd77e31000-7fcd77e65000 r-xp 0000a000 00:0e 23763188                   /opt/gtk/lib64/libnice.so.10.11.0
/proc/20888/maps:7fcd77e65000-7fcd77e7b000 r--p 0003e000 00:0e 23763188                   /opt/gtk/lib64/libnice.so.10.11.0
/proc/20888/maps:7fcd77e7b000-7fcd77e7c000 ---p 00054000 00:0e 23763188                   /opt/gtk/lib64/libnice.so.10.11.0
/proc/20888/maps:7fcd77e7c000-7fcd77e7d000 r--p 00054000 00:0e 23763188                   /opt/gtk/lib64/libnice.so.10.11.0
/proc/20888/maps:7fcd77e7d000-7fcd77e7e000 rw-p 00055000 00:0e 23763188                   /opt/gtk/lib64/libnice.so.10.11.0

So I wonder why that has changed. I still see nothing about a libsoup, but at least I have learned some more.

You can use the LD_DEBUG environment variable to get some more insight on which, when and from where libraries loaded. See eg ld.so(8) - Linux manual page for details. Also for deferred vs. immediate symbol resolution and binding

1 Like

Great hint. With latest libnice from git I get libsoup output, while my older libnice shipped with Gentoo-Linux shows no libsoup output for

$ LD_DEBUG=libs ./gen 1 2>&1 | grep soup
     28208:	find library=libsoup-2.4.so.1 [0]; searching
     28208:	  trying file=glibc-hwcaps/x86-64-v3/libsoup-2.4.so.1
     28208:	  trying file=glibc-hwcaps/x86-64-v2/libsoup-2.4.so.1
     28208:	  trying file=tls/haswell/x86_64/libsoup-2.4.so.1
     28208:	  trying file=tls/haswell/libsoup-2.4.so.1
     28208:	  trying file=tls/x86_64/libsoup-2.4.so.1
     28208:	  trying file=tls/libsoup-2.4.so.1
     28208:	  trying file=haswell/x86_64/libsoup-2.4.so.1
     28208:	  trying file=haswell/libsoup-2.4.so.1
     28208:	  trying file=x86_64/libsoup-2.4.so.1
     28208:	  trying file=libsoup-2.4.so.1
     28208:	  trying file=/opt/gtk/lib64/libsoup-2.4.so.1
     28208:	  trying file=/usr/lib64/libsoup-2.4.so.1
     28208:	calling init: /usr/lib64/libsoup-2.4.so.1
     28208:	./gen: error: symbol lookup error: undefined symbol: soup_date_time_new_from_http_string (fatal)
     28208:	calling fini: /usr/lib64/libsoup-2.4.so.1 [0]

So now I can understand what is going on, and I can try to fix it.

Maybe we have finally found a perfect solution to this issue.

First, the difference between the older libnice from Gentoo and the new libnice from Git is:

$ diff /usr/share/gir-1.0/Nice-0.1.gir /opt/gtk/share/gir-1.0/Nice-0.1.gir 
14c14
<              shared-library=""
---
>              shared-library="libnice.so.10"

So the Git libnice now specifies a shared-library string, which lets gobject-introspection now load all the other libs including libsoup. My guess is that gobject-introspection loads all the libs listed by ldd with dlopen.

But we may have a solution now: After finding the new location https://gnome.pages.gitlab.gnome.org/gobject-introspection/girepository/ I was again meditating about all the available functions, until I discovered https://gnome.pages.gitlab.gnome.org/gobject-introspection/girepository/gi-GITypelib.html#g-typelib-free.

And I realized that that function was never called at the end of function gintro/gen.nim at master · StefanSalewski/gintro · GitHub. Actually, when we wrote that code for Nim bindings generation seven years ago, we left out all the unref/free functions by intent.

But the fantastic news seems to be, that calling g_typelib_free() unloads all loaded symbols, so name conflicts for already existing symbols do not occur any longer. We intended to archive the same by processing each lib in a separate process, which is very difficult.

Well, we have to do more test, especially with the ArchLinux which generated most trouble, but it looks really promising now.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.