This style of tweak is typically provided by the compositor. The application just sets the dialog as modal and transient for the parent window, and then the compositor is expected to make appropriate changes to the appearance. That is how it works for gnome, at least.