The issue
This discussion is a follow up to the following issues:
- GTK https://gitlab.gnome.org/GNOME/gtk/issues/2025
- gnome-shell https://gitlab.gnome.org/GNOME/gnome-shell/issues/1471
The problem is that input events such as a touchpad “2-finger scroll” gesture are not synchronized to the screen refresh (‘vblank’) events. That results in jittery animation. Please refer to the GTK bug report above for example videos and detailed description.
As an example, my external Apple ‘trackpad’ V1 generates around 90 scroll events per second. That means that on average, on a 60FPS display we get 1 scroll event on the first frame, 2 events on the second frame, 1 on the third, 2 on the forth etc, and sometimes we get 2 events twice in a row. If, say, each event results in a 4px movement, we’ll get 4px on the first frame, 8px on the second, 4px on the third, 8px on the forth etc. This causes a visible jitter.
Example scenarios
Scenario 1
An external touchpad which sends 90 events per second is connected to a computer with a 60Hz display. Here we get 1 scroll event on the first frame, 2 events on the second frame, 1 on the third, 2 on the forth etc, and sometimes we get 2 events twice in a row.
Scenario 2
An internal touchpad with a poll interval of 125Hz is connected to a computer with 240Hz display. In this case an input event is only received every other frame, and once in a while it will be received 2 frames in a row.
Scenario 3
A standard USB mouse is connected to a monitor with a variable refresh rate, a.k.a ‘adaptive sync’, ‘free sync’ etc. Here we can get varying number of input events per frame.
Scenario 4
A standard 125Hz USB mouse is connected to a standard monitor with a 60Hz refresh rate. There will be a jank frame about 5 times per second. That’s because we’ll get on average 2 input events per display frame, but for each second there are 5 ‘extra’ input events (125 modulo 60).
The proposed solution
The input events should be resampled to the screen vblank times so that only a single, interpolated input event is processed in each frame. Absolute input events can be directly resampled whereas relative events should be converted to absolute events first, for example by calculating the cumulative sum of the deltas.
Show me the code…
Following is an excerpt from a POC implementing that solution for scroll gestures in GTK (the complete code can be found in https://gitlab.gnome.org/yarivb/gtk under the branch yariv-test-scroll
):
/* we have 2 points */
ScrollHistoryElem *first_elem;
ScrollHistoryElem *second_elem;
gdouble ratio;
first_elem = &g_array_index (scroll->scroll_history, ScrollHistoryElem, i);
second_elem = &g_array_index (scroll->scroll_history, ScrollHistoryElem, i + 1);
ratio = (gdouble)(interpolation_point - first_elem->evtime) /
(second_elem->evtime - first_elem->evtime);
interpolated_x = (ratio * second_elem->x) + ((1.0 - ratio) * first_elem->x);
interpolated_y = (ratio * second_elem->y) + ((1.0 - ratio) * first_elem->y);
/* synthesize the interpolated scroll event */
interpolated_item->dx = interpolated_x - scroll->total_x_offset;
interpolated_item->dy = interpolated_y - scroll->total_y_offset;
Here interpolation_point
is the frame time. Under GTK that is obtained from gdk_frame_clock_get_frame_time()
. The objects first_elem
and second_elem
hold 2 consecutive scroll history events. first_elem->{x,y}
and second_elem->{x,y}
are total offsets (sum of scroll deltas) from when the scroll gesture started.
We assume that the interpolation point lies between the 2 events, i.e. first_elem->evtime
< interpolation_point
< second_elem->evtime
. This excerpt implements a simple linear interpolation, though of course other interpolation methods could be used.