G_main_context_iteration and signal-safety(7)

The answer to that is:

  1. break down the loading into chunks
  2. use an idle handler to iterate through each chunk

Finding the size of each chunk to reduce the amount of blocking is left as an exercise for the reader—since it mainly depends on the amount of work each row has to do.

I wrote a simple tutorial for this particular pattern in 2006.

There’s also the problem that GtkTreeStore operations are accidentally quadratic and while there might be some possible solution to avoid that, it’s also not very interesting because GtkTreeStore has been deprecated in GTK4, and changing GTK3 comes with the potential of regressions.

2 Likes

To extract a few relevant points from that GTK issue:

  • If your data model means you can use a GtkListStore instead of a GtkTreeStore, then that will be faster
  • If not, consider implementing GtkTreeModel directly on top of your source data

Function that GTK uses internally have to be enclosed as well, like strftime() and malloc(). The internal state of these functions gets messed up when g_main_context_iteration() triggers one of them.

For now, it seems to work. If it crashes again, I’ll let you know.

If you are blocking the signal during GTK functions then you are just re-creating the same problem but in a very roundabout sort of way. Because then the signal handler will not be invoked if a GTK function blocks for a long time and it will still be delayed. And to be totally safe, you have to put sigprocmask calls around every single non-async-signal-safe function including every single GTK function call. This is in addition to the normal logic issues you can have when trying to run a nested inner loop like that. So it is probably more work in the long run for you to do what you’re doing, than just fixing the app to correctly load the data in a non-blocking way.

This is not something you can “win”, there is no point to trying to outsmart a library. I suggest you use things in the intended way and don’t try to come up with these “clever” hacks, you will thank yourself later…

I’m starting to think you do not understand the concept of async-signal-safety. You really need to study that carefully before messing with Unix signal handlers.

Ok, I believe you.

If you place sigprocmask() at the beginning and the end of every outer loop (the GtkTreeStore rows) the result is worse than a manual gtk_main_context_pending() + gtk_main_context_iteration() at the same position.

Apparently I forgot about one thing, the kernel scheduler doesn’t necessarily get the opportunity to start the timer signal handler in between the end of one iteration and the start of the next. The solution would have been a sigpending() that starts that handler automatically when a signal is pending, but it doesn’t.

You’re probably talking about the sorter? This is not by accident, sorting costs (n-0)+(n-1)+(n-2) … comparison operations. There’s nothing that can be done about it, nor can it be done in parallel. (It is also not a reason to remove the complete GtkTreeStore.) Why are you all so negative about GtkTreeStore and related?

Oops… counting in the rows at other depths. Although the number of sort_funcs I count is not constant (1852->1841) I needed 642411 comparison operations to sort that many rows. Never mind :slight_smile:

Does someone know what the exact conditions for a successful call to g_main_context_pending(NULL) / g_main_context_iteration(NULL, FALSE) are?

My application crashes with a SIGSEGV when these functions are called and I think when the main window is not yet visible. I already tried to check for (g_main_context_default() != NULL), but that apparently not enough.

There are none, that function is simply not async signal safe.

No man, I’m talking about the manual insertions, not those by a signal handler.

Looks pretty insane, a ‘if (g_main_context_pending(NULL)) g_main_context_interation(NULL, FALSE);’ in a ‘gtk_tree_iter_compare_func’ comparison operation, but it works.

For some reason, it did make the program crash at certain points (while loading a GtkGLArea). I’ve replaced all 'while (g_main_context_pending(NULL))'s with 'if (g_main_context_pending(NULL))'s and now it works, but I can’t really explain why it crashed after three iterations.

Also, I haven’t found a way to attach a progress bar to the sorting algorithm, any ideas? There aren’t any progression signals emitted by the sorter, are there?

Sorting should be O(n*lg(n)). It should not be quadratic, O(n^2). Look up the “merge sort” or “quick sort” algorithms for easy examples of non-quadratic sorting.

It’s deprecated because it’s complicated and slow. You’ll have to get rid of it eventually when you upgrade to GTK 5. If you have existing software using it, that’s fine, but writing new software that uses it is not advisable.

Of course. g_main_context_pending() is safe to call at any time. g_main_context_iteration() is safe to call whenever the main context is not being iterated (except, of course, when async signal safety is required). In a GTK application, you should never call g_main_context_iteration() because GTK is already iterating the main loop for you. Iterating a main context that is already being iterated would result in a nested iteration, which frequently leads to errors and bugs because application developers do not expect callbacks to execute before the current main context iteration completes. So stop trying to do it: it’s an anti-pattern, and there’s a decent chance that’s responsible for the crashes you encounter when you try it. (Beware that in GTK 3, gtk_dialog_run() actually does this internally, so I recommend avoiding that function as well.)

No, I’m not talking about sorting. The link I posted explains precisely the quadratic behaviour on insertion, and why it’s not trivial to fix without the potential of a regression.

Because it’s an old API, designed for another time, and it’s been deprecated for a reason.

Well, I don’t think I have much choice here. I already explained that the handler of g_timeout_add() (a callback) is not being executed while we’re inside a GtkWidget signal handler like GtkTreeView::cursor-changed. Because my cursor-changed changes the GtkTreeModel of a second GtkTreeView (and has to load its GtkTreeStore when needed), the handler potentially takes a long time to finish (32 threads - 10 minutes). That’s why I use a GtkProgressBar to show the progress while loading the GtkTreeStore. The problem is that the main context (main loop) is not iterated while we’re inside the GtkTreeView::cursor-changed handler, I have to iterate it manually to make the GtkProgressBar work. In other words, I can set fractions as many as I like, but the GUI won’t get updated when I don’t iterate the context manually.

And why does it hop from 52 to 41 iterations? Because 18*ln(18) is indeed 52.

Again, don’t iterate manually. It would be much better to fix your code so the cursor-changed signal doesn’t take 10 minutes to return. Several choices of how to do this have been mentioned previously.

Edit: If your work threads are blocking the main loop then you are implementing work queues incorrectly, the purpose of using them is to avoid that. Really what you likely want to do in a database application, if you retain the connection to the database, is to make a custom GListModel backed by the database and switch to GtkColumnView. You can still have the fetching done in other threads so the app will automatically lazy load rows as necessary. You can also have the sorting done in the query so as to use any db indexing to speed up the sort.

That’s the best solution I heard so far!

My code might not be perfect, but at least it works and it doesn’t crash. As long as I don’t get the Linux signals to do what I need them to do, this will do perfectly.

During this play with the GtkTreeVIew widget, I found a bug or two that I was planning to fix later on. I suggest that one of you tries to fix them, unless you’re willing to wait for me.

Again: you can’t get them to do that. GTK is not designed to work with signals in that way.

Jason,

I am a sucker for things that can’t be done.

I too wish that unicorns, elves and wizards were real, but sadly they are not…

1 Like