The way your image appears, it looks like you added the buttons and then added the overlay with the stack over the widget that contains the buttons. Since, to achieve the behavior you want, the overlay should contain both the buttons and the stack. As the overlay can only contain one child, the stack and the buttons would probably be inside a Gtk.Box that would be the child of the overlay.
Aah, I think this clicked something. Basically I was adding the buttons and stack as overlays and thought the overlay container keeps track of widgets added through add_overlay as it’s children so the stack should know about the buttons when it actually doesn’t I guess. Overlay widgets are just that, overlays. They don’t know anything about other overlay widgets added in the overlay container.
I managed to circumvent the issue by adding a centerbox as an overlay widget through add_overlay. So now I can still use the overlay container, add a center box for the main UI and then still add the menu bars and tool icons as overlay widgets on top of the center box.