Nautilus does not generate the thumbnails itself but uses a thumbnailer to generate them. Which thumbnailer gets used depends on what you have installed. In a standard gnome setup this would be totem-video-thumbnailer, which should support extracting cover images (or other image tags) from videos.
You can check /usr/share/thumbnailers which thumbnailers are used for which mime type. Maybe you are using a different thumbnailer that does not support this.
Or maybe there is a bug in totem (or somewhere at a lower level, such as in gstreamer) that would cause it to not find the included thumbnail. You can try running totem-video-thumbnailer manually to check if it correctly extracts the included thumbnail. If it doesn’t I would suggest filing a bug report against totem: Issues · GNOME / totem · GitLab