Drawing area + scroll memory problems

[molnarandris (András Molnár)]

Dear all,

I am trying to make a latex editor in gjs; as a part of it, i’m trying to display pdf files. So far what I’m doing is to create a drawing area inside a scrolled window, set the size of the drawing area to the size of the whole document (such that the scrolling works), and connect to the draw signal to repaint the pdf with Poppler when scrolling. However, this process seems to eat a lot of memory (a few GB after scrolling through a 10-page document a few times). I guess I’m doing something wrong; I have managed to find a minimal non-working example for my problem (at least I hope it’s the same problem):

#!/usr/bin/gjs

imports.gi.versions.Gtk = "3.0";
const {Gtk} = imports.gi

Gtk.init(null)

let width = 400 
let height = 600

let area = new Gtk.DrawingArea();
area.set_size_request(width, height);
area.connect('draw', (area, ctx) => {
});

const scroll = new Gtk.ScrolledWindow({ hexpand: true, vexpand: true });
scroll.add(area);
const win = new Gtk.Window({ defaultHeight: height, defaultWidth: width });
win.connect('destroy', () => { Gtk.main_quit(); });
win.add(scroll);
win.show_all();

Gtk.main();

When executing this code and resizing the window like crazy for a few minutes, the memory usage rapidly increases (and never decreases after). The closest thing related I found is this bug in Ruby, but here I have no problem without the scroll area.

So my question is: am I doing something wrong even in this tiny example or is this a bug?

Thanks a lot.

P.S. I have given up on Evince due to the lack of examples – I couldn’t figure out how to make synctex work, more specifically, how to jump to a given coordinate in the pdf and then highlight a given area.

[ebassi (Emmanuele Bassi)]

Your draw signal implementation is incorrect. You need to explicitly release the reference on the Cairo context you are given—which is the cause of your leak; and you need to return a value from the signal handler—as the GtkWidget::draw signal signature has a boolean return value.

The correct handler is:

area.connect('draw', (area, ctx) => {
    ctx.$dispose();
    return true;
});

See the corresponding page in the GJS documentation.

[molnarandris (András Molnár)]

Ahh, great, thanks a lot! Thanks for the link to gitlab too, I wasn’t aware of those pages.

[molnarandris (András Molnár)]

Thanks again for the help! At the end I think my original memory leak is that in the code

let doc = Poppler.Document.new_from_file(path,null);
let page = doc.get_page(0);
area.connect('draw', (area, ctx) => {
    page.render(ctx);
    ctx.$dispose();
    return true;
});

when page is set to a new value, the original content still stays in the memory (if it got rendered). What seems to solve this is to import the system module and run the garbage collector system.gc() whenever page gets overwritten.

[ebassi (Emmanuele Bassi)]

You could use let page = doc.get_page(0) inside the signal handler, so that you don’t have a reference across scopes keeping the variable alive.

[molnarandris (András Molnár)]

Thanks for the answer! Am I doing then again something very stupid in the code below (apart from being super inefficient)?

#!/usr/bin/gjs

imports.gi.versions.Gtk = "3.0";
const {Gtk, Poppler} = imports.gi
const System = imports.system;

Gtk.init(null);

let path = 'file:///path/to/pdf';
let doc = Poppler.Document.new_from_file(path,null);
let n   = doc.get_n_pages();
let [width,height] = doc.get_page(0).get_size();
const area = new Gtk.DrawingArea();
const scroll = new Gtk.ScrolledWindow({ hexpand: true, vexpand: true });
const win = new Gtk.Window({ defaultHeight: height, defaultWidth: width });

area.set_size_request(width, n*height);

scroll.add(area);
win.add(scroll);

area.connect('draw', (area, ctx) => {
  for (let i=0; i<n; i++){
    let page = doc.get_page(i);
    page.render(ctx);
    ctx.translate(0,height);
  }
  ctx.$dispose();
  //System.gc();
  return false;
});

win.connect('destroy', () => { Gtk.main_quit(); });

win.show_all();
Gtk.main();

With this code scrolling through a 20-page document (up and down) eats 1.5GB memory for me. After uncommenting System.gc() it’s down to 40MB.

In my current code – unlike above – I don’t use get_page in the callback, as I’m not sure how expensive it is (in computation power), but have to reload the pdf from time to time, and then the memory still leaks without explicit garbage collection.

[ebassi (Emmanuele Bassi)]

You’re rendering every page, which can be very expensive to do; you probably want to know where you are in the document depending on the scrolled window’s adjustment position, and only render the visible pages. Another trick is to do pagination instead of continuous scrolling, but that kind of depends on the UI.

You also have to remember that Cairo (and the windowing system surface you’re rendering to) has a hard limit on the surface size—which could be 16k pixels or 32k pixels depending on the backend in use. This means you should definitely limit the rendering surface, especially for large documents; pagination is most helpful, there.

In general, it’s not just JavaScript’s GC collection that will help you: it’s literally not doing work to begin with.

[molnarandris (András Molnár)]

Great, thanks for the advice! I’ll definitely try to implement some kind of pagination.

[molnarandris (András Molnár)]

Sorry for the re-post, but I still don’t understand what is going wrong here. I think I wasn’t clear, my problem is not the amount of memory that the scrolling takes (I know that it’s extremely inefficient what I’m doing), but the fact that the memory usage continuously grows. The 1.5GB for scrolling up and down becomes 3GB after doing it twice, and 6GB doing it four times, etc.

This is not the behavior I expected naïvely, as when page goes out of scope, I would expect the memory be freed after it. It looks like, however, that if page gets rendered, that is not the case anymore: without explicit garbage collection, every page that I render stays in the memory forever.

I might be completely wrong, but if I had to guess, I would say that if I render the page, ctx will hold a reference to it, and thus it can’t be removed from the memory when it leaves the scope. But then shouldn’t ctx.$dispose get rid of that reference?

[ptomato (Philip Chimento)]

Can you verify whether garbage is actually being collected in between scrolls? The problem is that the garbage collector isn’t aware of the potentially large amount of memory that Cairo objects take up, so if you have a lot of them then it doesn’t necessarily realize that it’s time to do a garbage collection.

Run your app with sysprof and you should be able to see timing marks on the graph when the garbage collector runs.