I cannot remove Geary, I have unistalled it through the software center, deleted the files in .config and .shared, and apt-get purge geary, amongst other terminal commands. Nothing has worked yet.
However every time I reinstall Geary, the icon appears in the exact position it was on the tray before removal, and all my old accounts are still there.
How do I get this thing removed completely off my system so I can do a clean install without any of my old accounts & settings?
Most likely then settings in question are not part of Geary.
By the “tray” I suspect you mean the dash, in which GNOME Shell shows the favorited applications? This is part of GNOME Shell, and therefore the setting is part of it.
Also, Geary can use accounts which are set up using GNOME Online Accounts, which are then saved by the Evolution Data Server, outside of Geary’s settings. You can check if there are set up accounts in the online accounts page in the control center.