Give players the option to vote to swap server in the crew management tab while maintaining current voyage, supplies, loot, and emissary status.
The mechanic is already built into the game for when players end up on an empty server. Just give us the option to trigger it ourselves.
The option can't be exploited in any way that players haven't already figured out another way to do. And it'll give players a way to escape griefers who chase you around the map and prevent you from completing voyages.
I don't mind the occasional encounter with other pirates but it gets old when you need to constantly deal with players who don't care about voyages and rep and only want to mess with people.
