Hi,
Over the last 24 hours, the GNOME Infrastructure Team responded to a major traffic anomaly targeting our public frontend infrastructure.
The Timeline
An initial anomaly triggered an upward trend on May 23 at 21:00 EST, which escalated into a total OVN control plane failure by 01:30 EST, on May 24.
At its peak around 02:00 EST, the load balancer metrics recorded a massive surge, topping out at a sustained rate of 1.42M concurrent requests per second. The primary target was www.gnome.org.
Impact on OVN and Nodes
The OVN-Kubernetes control plane and individual kubelet daemons were impacted on nodes running the pods serving www.gnome.org traffic. As a direct result, several affected worker nodes temporarily drifted into a NodeNotReady state, triggering localized pod evictions and other services outages.
Next Steps
We are continuing to monitor the cluster closely. We’re evaluating additional next steps including moving GitLab pages to Fastly.
If you notice any anomaly, please open a ticket on our Infrastructure tracker on GitLab.