Proper zoom/pan image approach for large images

Hello GTK Enthusiasts,

Our group is working with electron microscope images that are commonly 25,000 x 25,000 (very large). We’ve written a Java application that allows zooming and panning to any resolution. It is implemented by figuring out where to draw the upper left corner of the image and at what scaling. The upper left corner is typically far far away, but the Java Virtual machine does a very good job of figuring out what section actually appears in the window and just drawing that portion. It is amazingly fast.

We have ported it to Python2 with GTK2 (pygtk), and it worked but got very slow (eventually locking) as we zoomed in on even relatively small images. We used the “draw_pixbuf” API function to do the drawing. We’d like to move to GTK3 (in either Python3 or C), and we’d like to know if there’s a better function to use for drawing portions of large images into a GTK3 window (DrawingArea or other?). Would it be best to do it in GTK itself or should it be done through cairo? We’re relatively new to both, so any examples would help.

Thanks in advance.

What kind of pictures are you taking with the electon microscope? Tranmission or scanning?

That is a good sized picture and I tried a 25,000 x 25,000 surface and got an out of memory error. My computer isn’t the newest. If you can read the picture into a cairo surface then you could partition it and then scale it before drawing. I tried a 5000 x 5000 test surface and that works fine. Maybe the same idea will work with a bigger surface and newer hardware.

Eric

/*
    gcc -Wall big_surface1.c -o big_surface1 `pkg-config --cflags --libs gtk+-3.0`

    Tested on Ubuntu18.04 and GTK3.22
*/

#include<gtk/gtk.h>

static gdouble translate_x=0.0;
static gdouble translate_y=0.0;
static gdouble scale=1.0;

static cairo_surface_t* big_surface_new();
static void translate_x_spin_changed(GtkSpinButton *spin_button, gpointer data);
static void translate_y_spin_changed(GtkSpinButton *spin_button, gpointer data);
static void scale_spin_changed(GtkSpinButton *spin_button, gpointer data);
static gboolean da_drawing(GtkWidget *da, cairo_t *cr, cairo_surface_t *big_surface);

int main(int argc, char **argv)
 {
   gtk_init(&argc, &argv);

   GtkWidget *window=gtk_window_new(GTK_WINDOW_TOPLEVEL);
   gtk_window_set_title(GTK_WINDOW(window), "Big Surface");
   gtk_window_set_default_size(GTK_WINDOW(window), 500, 500);
   gtk_window_set_position(GTK_WINDOW(window), GTK_WIN_POS_CENTER);
   g_signal_connect(window, "destroy", G_CALLBACK(gtk_main_quit), NULL);

   //Get a test surface.
   cairo_surface_t *big_surface=big_surface_new();

   GtkWidget *da=gtk_drawing_area_new();
   gtk_widget_set_hexpand(da, TRUE);
   gtk_widget_set_vexpand(da, TRUE);
   g_signal_connect(da, "draw", G_CALLBACK(da_drawing), big_surface);

   GtkWidget *scroll=gtk_scrolled_window_new(NULL, NULL);
   gtk_widget_set_hexpand(scroll, TRUE);
   gtk_widget_set_vexpand(scroll, TRUE);
   //Enougth drawing area size to scale the partition.
   gtk_widget_set_size_request(da, 1000, 1000);
   gtk_container_add(GTK_CONTAINER(scroll), da);

   GtkAdjustment *translate_x_adj=gtk_adjustment_new(0.0, 0.0, 5000.0, 50.0, 0.0, 0.0);
   GtkAdjustment *translate_y_adj=gtk_adjustment_new(0.0, 0.0, 5000.0, 50.0, 0.0, 0.0);
   GtkAdjustment *scale_adj=gtk_adjustment_new(1.0, 0.2, 2.0, 0.1, 0.0, 0.0);

   GtkWidget *translate_x_label=gtk_label_new("translate x");
   GtkWidget *translate_x_spin=gtk_spin_button_new(translate_x_adj, 50.0, 1);
   g_signal_connect(translate_x_spin, "value-changed", G_CALLBACK(translate_x_spin_changed), da);

   GtkWidget *translate_y_label=gtk_label_new("translate y");
   GtkWidget *translate_y_spin=gtk_spin_button_new(translate_y_adj, 50.0, 1);
   g_signal_connect(translate_y_spin, "value-changed", G_CALLBACK(translate_y_spin_changed), da);  
    
   GtkWidget *scale_label=gtk_label_new("Scale");
   GtkWidget *scale_spin=gtk_spin_button_new(scale_adj, 0.2, 1);
   g_signal_connect(scale_spin, "value-changed", G_CALLBACK(scale_spin_changed), da);  

   GtkWidget *grid=gtk_grid_new();
   gtk_grid_attach(GTK_GRID(grid), scroll, 0, 0, 3, 1);
   gtk_grid_attach(GTK_GRID(grid), translate_x_label, 0, 1, 1, 1);
   gtk_grid_attach(GTK_GRID(grid), translate_y_label, 1, 1, 1, 1);
   gtk_grid_attach(GTK_GRID(grid), scale_label, 2, 1, 1, 1);
   gtk_grid_attach(GTK_GRID(grid), translate_x_spin, 0, 2, 1, 1);
   gtk_grid_attach(GTK_GRID(grid), translate_y_spin, 1, 2, 1, 1);
   gtk_grid_attach(GTK_GRID(grid), scale_spin, 2, 2, 1, 1);
   
   gtk_container_add(GTK_CONTAINER(window), grid);

   gtk_widget_show_all(window);

   gtk_main();

   //Clean up.
   cairo_surface_destroy(big_surface); 

   return 0;  
 }
static void translate_x_spin_changed(GtkSpinButton *spin_button, gpointer data)
  {
    translate_x=gtk_spin_button_get_value(spin_button);
    gtk_widget_queue_draw(GTK_WIDGET(data));
  }
static void translate_y_spin_changed(GtkSpinButton *spin_button, gpointer data)
  {
    translate_y=gtk_spin_button_get_value(spin_button);
    gtk_widget_queue_draw(GTK_WIDGET(data));
  }
static void scale_spin_changed(GtkSpinButton *spin_button, gpointer data)
  {
    scale=gtk_spin_button_get_value(spin_button);
    gtk_widget_queue_draw(GTK_WIDGET(data));
  }
static cairo_surface_t* big_surface_new()
  {
    gint i=0;

    //Use gdk_cairo_surface_create_from_pixbuf() to read in a pixbuf. Try a test surface here.
    cairo_surface_t *big_surface=cairo_image_surface_create(CAIRO_FORMAT_ARGB32, 5000, 5000);
    cairo_t *cr=cairo_create(big_surface);

    //Paint the background.
    cairo_set_source_rgba(cr, 1.0, 1.0, 1.0, 1.0);
    cairo_paint(cr);

    //Draw some test grid lines.
    cairo_set_source_rgba(cr, 0.0, 1.0, 0.0, 1.0);
    for(i=0;i<50;i++)
      {
        cairo_move_to(cr, 0.0, (gdouble)i*100.0);
        cairo_line_to(cr, 5000.0, (gdouble)i*100.0);
        cairo_stroke(cr);
      }
    for(i=0;i<50;i++)
      {
        cairo_move_to(cr, (gdouble)i*100, 0.0);
        cairo_line_to(cr, (gdouble)i*100, 5000.0);
        cairo_stroke(cr);
      }

    cairo_set_source_rgba(cr, 0.0, 0.0, 1.0, 1.0);
    cairo_set_line_width(cr, 10.0);
    for(i=0;i<10;i++)
      {
        cairo_move_to(cr, 0.0, (gdouble)i*500.0);
        cairo_line_to(cr, 5000.0, (gdouble)i*500.0);
        cairo_stroke(cr);
      }
    for(i=0;i<10;i++)
      {
        cairo_move_to(cr, (gdouble)i*500.0, 0.0);
        cairo_line_to(cr, (gdouble)i*500.0, 5000.0);
        cairo_stroke(cr);
      }

    //Outside box.
    cairo_set_line_width(cr, 20.0);
    cairo_set_source_rgba(cr, 1.0, 0.0, 1.0, 1.0);
    cairo_rectangle(cr, 0.0, 0.0, 5000.0, 5000.0);
    cairo_stroke(cr);

    cairo_destroy(cr);

    return big_surface;
  }
static gboolean da_drawing(GtkWidget *da, cairo_t *cr, cairo_surface_t *big_surface)
 {
   gdouble origin_x=translate_x;
   gdouble origin_y=translate_y;
   
   //Some constraints.
   if(translate_x>4500.0) origin_x=4500.0;
   if(translate_y>4500.0) origin_y=4500.0;

   cairo_set_source_rgba(cr, 0.0, 0.0, 0.0, 1.0);
   cairo_paint(cr);

   //Partition the big surface.
   cairo_surface_t *little_surface=cairo_surface_create_for_rectangle(big_surface, origin_x, origin_y, 500.0, 500.0);

   cairo_scale(cr, scale, scale);
   cairo_set_source_surface(cr, little_surface, 0.0, 0.0);
   cairo_paint(cr);
   
   cairo_surface_destroy(little_surface); 
   return FALSE;
}

For very large images, I’d recommend using Gegl. Cairo has limits on the size and precision of its surfaces, so you’d have to implement fairly complex tiling code to handle panning and zooming. Gegl is using by photo and image editors, like GNOME Photos and the GNU Image Manipulation Program, and provides a tiled buffer and various facilities that make implementing viewers a lot easier.

This is one of the reasons that I wrote the gtk_image_viewer widget a long time. When zooming a part of an image it only zooms the part of the image that shown on the screen. In case an image is too large to be shown in memory, you can also generate the image to be shown “on the fly” in the “annotation” callback. One of the examples included with the widget is a Mandelbrot image that can be panned and zoomed which is generated in the “annotation” calllback. I also have an code somewhere (though it is not part of the widget examples) that allowes displaying huge one-bit uncompressed image, where I random seek the image on disk to retrieve only the rectangle that is currently displayed.

Please let me know if this sounds interesting, and I’ll see if I can clean it up enough for general purpose use.

See: http://giv.sourceforge.net/gtk-image-viewer/

Thanks for all the suggestions and help.

What kind of pictures are you taking with the electon microscope? Tranmission or scanning?

I think they may be both. I’m not an expert in the data collection, and to me they’re just very large images. We are looking at brain tissue, but other uses are possible.

After a number of failed attempts, I decided to see how well the images are handled in GIMP itself. GIMP seems to zoom these large images very well, but panning was surprisingly clunky when the large images were zoomed in. In those cases, large blocks of pixels could be seen being drawn in chunks. I wouldn’t say it was unusable, but it wasn’t nearly as smooth as the Java version. This concerns me because I would assume that GIMP is doing the best that can be done with GTK (and in C). I have run the gtk-image-viewer (thanks), but I haven’t adapted it to handle our specific images. Does anyone know if it is likely to be more efficient than GIMP itself?

By contrast, in the Java version I’m simply calling this one function:

g.drawImage ( image, x, y, w, h, this );

The x and y values tend to be very large negative values (placing the upper left corner far off the screen). The w and h values are the width and height of the image itself (tending to place the lower right corner far off the screen as well). As long as I keep track of those values, the Java runtime will clip out the pixels it needs to fit the drawing area and render them as zoomed and panned. It’s very easy and amazingly fast.

Is there any hope that GTK might provide similar functionality in the near future? Or is there a reasonably easy way to “reach around” GTK to get higher performance while still maintaining cross-platform portability?

Thanks in advance.