How to load objects on a background thread

Platform Language bindings
,
1

I’m doing some heavy loading and a few calculations for my GTK application. To not stall the event loop, I have to move these on to a separate thread. I know I can do a few things on the event loop using async but this only got me so far and was still slowing everything down (my calculations were slow because of the constant interruptions and the animation was still choppy because of the calculations).

The main problem boils down to the fact that objects like gdk_pixbuf::Pixbuf or poppler::PopplerDocument are not Send. I understand that this is not a simple problem because the structures are (non-atomically) reference counted. Here’s the solutions I’ve already come up with:

  • Do everything on one thread in background tasks using async.
  • Use unsafe and send the object over anyway if I know I’m the exclusive owner. YOLO!
  • Serialize all data into a vec and then deserialize on the other side. (YMMV depending on the data, but images and PDF documents should work fine.)

None of these feels good. I’d really need some “official”, supported and built-in mechanism to somehow do these things with less pain. Some random ideas off the shelf:

  • Provide some owned variants for all objects without that reference counter.
  • Create a safe Send wrapper that fails if the data has RC > 1.
  • Create a safe Send wrapper that deep-clones the data where needed to ensure ownership
  • Make the reference counters atomic and then add some mutex mechanism (Probably not a realistic one).
2

They are actually atomically refcounted. The problem is that you can’t have non-threadsafe interior mutability, reference counting and Send at the same time. This is also why std::sync::Arc only implements Send if the contained type is Sync.

In short, GLib-style APIs use reference counting too much where they really shouldn’t and as a result many things are hard to use in a language that statically ensures thread-safety. And reference counting is used so much because it makes life easier in C and generally in languages that have no concept of borrowing and thread-safety.

async probably doesn’t solve your problem. Any multithreaded async executor requires all its futures to be Send, which will result in the same problem again.

Any singlethreaded async executor (like gio::MainContext) won’t require that but then you would have to do the calculations on your main thread, which kind of defeats the purpose.

That would be my choice in your situation :slight_smile: Make sure to wrap that behind a small abstraction that makes it hard to accidentally misuse it. I think librsvg also does such things.

I’d possibly do a variation on this. Only handle raw data across the threads (Vec<u8> etc) and only construct the pixbuf/etc around that raw data when needed, and retrieving your original raw data out of it again when done. That’s not too easy either because retrieving the data without copying from reference counted types is problematic again. See the cairo threading example in the gtk-rs repo: it solves the same problem around cairo image surfaces.

GStreamer does such things where it makes sense. It’s some manual work though: you need to ensure that nothing else can ever get a reference to the contained value, and you need to get the mutability of everything right unless you want to continue relying on interior mutability (in that case: only Send, no Sync but that seems sufficient here).

There was some experiment around that in the glib bindings, which then allows this wrapper.

The trait has to be manually implemented for types where it makes sense, and you need to be very careful again about the API of the type. See the documentation in there.

That has all the problems above, plus some additional work. But ideally types like Pixbuf were implemented like this to begin with, then such problems wouldn’t exist :slight_smile:

That’s basically what GStreamer does with GstMiniObject and std::sync::Arc do (make_mut() / get_mut()): copy-on-write.

1 Like
3

Thanks for all the insights. I’ll probably just use the unsafe solution in the short term, as it is the easiest to implement, but I’d still like to discuss some more “proper” fixes for this.

I think librsvg also does such things.

Do you have a link/snippet, please?

I really like the approach of the UniqueAdapter, and I’d be happy if there existed similar constructions for selected Gtk classes. Once created, do those wrappers cause a lot of maintenance overhead? Would it even be possible to automatically generate them (or parts of them)?

The SendUnique approach looks very nice too, but unless implemented for any structs it is basically useless. Also I don’t think I can implement it myself locally due to the orphan rule :frowning:

Which of the approaches do you think have the most potential to actually land one day in the bindings?

4

I don’t, maybe @alatiera or @federico can point you to that. I remember it being related to cairo image surfaces there.

With some additions to the code generator those could be autogenerated. The big amount of work afterwards then is analyzing for each type if and how it can be enabled and mutability rules of the functions.

Generally all this would be much less of a problem if GObject was using C++ instead of C and would consider const-correctness a goal, but that’s not where we are unfortunately :slight_smile:

Well, someone will have to analyze if it can be enabled for Pixbuf, for example. And then it can be enabled accordingly in the bindings :slight_smile:

5

I gave some ideas a try and uploaded my current (partial) implementation as a gist. I’ve tried a few things and different styles, and also skipped methods that I didn’t really know how to wrap. Some things I’ve learned through this:

  • Owned wrapper types are great in a lot of ways
  • Some of the time, the API is easy to wrap, and this could be automated with some macros
  • But one really sees how the API was designed with reference counting and all of this in mind. Many operation thus accept a destination Pixbuf or simply return a copy of the data. This is not something one can really “wrap”, but rather has to be re-implemented in a more idiomatic way.
  • The subpixbuf is a tricky example, because one has to create a new struct that borrows form OwnedPixbuf in order to make the lifetimes work.
  • I also struggled with the from_stream-constructors, because they look like it’s async in the background so I can’t really tell when I’m the full exlusive owner of the data.
  • The owned wrapper type is a bit sad to use without being able to implement clone. We really need a deep cloning operation for objects. But if we had one, we could simply use the clone-and-send approach and don’t bother with all of this.
1 Like
6

Some follow up questions:

  • Because all reference counters are atomic, if my (wrapper) type does not expose any (interior) mutability, is it then Send/Sync? If not, why not?
  • What’s the state of the art for deep cloning glib objects? I really need this otherwise my type is pretty much useless (I cannot pass references of the inner value to Gtk without unwrapping my wrapper, but then I’m pretty much back to square one).
7

That’s correct as long as you don’t expose the reference counting (e.g. by having a Clone impl), see for example gstreamer_base::UniqueAdapter. Explicit mutability is fine FWIW.

And another thing to keep in mind is that various GTK types (basically all widgets and related GDK types) are bound to their thread because the underlying system resources can only be used from that single thread. Those can never be Send.

I’m not aware of anything there other than what exists for GstMiniObject, but yes then you’re back to square one :slight_smile:

1 Like
8

Argh, this is so frustrating ^^

The only way for an owned wrapper to be useful (which implies a safe abstraction) is if all components that accept it in some form or another are modified to also accept a borrow to the owned one. (The only other way is to deep clone and unwrap the inner in those cases, but yeah cloning.)

Would it be possible to implement a deep_clone method on the upstream side of things (or a make_unique, should be comparable)? How would I go about if I wanted to implement SendUnique for some type?

9

Sure, but that’s something to discuss in the issue tracker of the corresponding project then.

What would it help you here, and which part of implementing the trait is not clear to you?

10

What would a “deep clone” even look like for a GtkWidget? A clone of the entire widget hierarchy up to the top-level? Aside for the tons of memory wasted, it would also not solve anything, because the underlying windowing system resources would not be clonable, or be accessible from different threads.

Some object types might have a clone, but those are usually not interesting because they are plain data containers with some fancy API around them.

11

For something like GdkPixbuf it potentially makes sense, but it doesn’t seem like it would solve any of the problems here.

12

Well, I need to access the internals of the pixbuf, including all the reference counters and sub-objects. I guess I need to get a GdkPixbuf from the sys crate first, but then I’ve looked at the struct and it’s opaque. And from there I don’t know any further.

As widgets are not thread safe by definition, they are out of scope for my attempts. So I don’t really care about cloning them. So the types I care about are precisely those plain data containers you call “not interesting”. Pixbuf, PopplerDocument, maybe even some Cairo surfaces, you name it. Because from my Rust perspective, I think they are. And to keep the scope manageable, I mostly care about sending these to other threads at the moment, not about sharing them between thread.

13

Then you can’t implement it :slight_smile:

14

Well, if I can’t implement it for Pixbuf, then I doubt that can be implemented for anything at all (at least from the Rust side). And apparently my chances of implementing this upstream are rather low, but that’s fine.

I think I’ve finally made my piece with this issue and will simply use the unsafe-send-sync crate from now on when I don’t see any other way.

15

It can be implemented on GstAdapter at least. You need a way to check the refcount (can be done for all GObjects), and you need to be able to control references.

16

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.