So this is what I'd do.
PvE servers. It's super controversial, but hear me out. There are 3 types of players. Sheeps, boars, and wolves. Wolves prey on boars and sheep, boars prey on sheep, and sheep just get preyed on. They occasionally are dangerous to the group above them, but for the most part sheep are the most vulnerable, and numerous, group of gamers out there. Sheep need a fence. Boars also want, but do not need, a fence. You then want to incentivize players to come out of the fence. Without a fence, sheeps leave the game, which means the game in general doesn't grow its playerbase, a longstanding problem of Sea of Thieves.
So for PvE servers, I'd do these mechanics: No player can attack any other player and do damage for any reason. Just all friendly fire and player damage is turned off. Environmental effects such as kraken, skeleton ships, ghost ships, and so on are ramped up in lethality. PvE players want a PvE challenge, so make it more dangerous to sail around. Allow multiple krakens and megs to be up at a time, attack ships more often, if players aren't going to be a danger, then the game is the danger, make it dangerous.
This would be coupled with no stealing of loot. A player never has to worry about another player stealing their time. Ever. Except.....
Reapers would still exist. 1 player per server in PvE may raise a reaper's emissary flag. If you raise this flag, you can kill anyone and anyone can kill you. You can see anyone, anyone can see you. You still cannot steal loot, but if you sink a ship, a special crate spawns containing the "memory" of the ship you killed, which is empowered based on the time that ship has been alive, and when turned in is worth a tidy sum. Simultaneously, you also gain a boost as a reaper on a PvE server.
You can shoot cannonballs endlessly, you have endless stockpiles of wood, you cannot be killed from enemies boarding you, and are empowered when boarding enemies, you are now a boss, a pirate lord, cursed to forever be on your ship. Except when you set foot on reaper island or swim in the waters to steal the memory of fallen ships. And, of course, everyone wants to kill you because if they kill you, you lose your reaper status and as a new mechanic, everyone involved in sinking a ship instantly gets a substantial money, doubloon, and emissary reward, and reapers on PvE servers are the best reward, for both the reaper, and the reaped. Oh, and just for good measure, the reapers on PvE servers get their ship changed into a skeleton sloop, brig, or galleon, with a skeleton crew to man it.
From such a boss event, players are encouraged to PvP, but the only people who take on risks are the instigators who simultaneously also get a lot of bonuses, while getting to have a lot of fun. None of the reaped can lose their loot. Even if sunk, it stays with them when their ship respawns, nor can players not of the crew pick up the loot.
The second half of this is "What happens to PvP servers?" As I said before, you both need a fence, and a reason to cross over the fence. So PvP servers instead get massive bonuses to all rewards. Think 100% increase to all emissary, gold, and doubloon rewards. You should progress faster on a PvP server, and without the threat of AI monsters constantly attacking you as it would be on a PvE server, it should be the easier method, until a player shows up.
And as I brought up on PvE servers, the act of sinking an enemy ship should reward you by itself, in reaper rep and gold. But PvP servers let you claim the grim bounty as they do now.
However, all outposts, except reaper island, now are safe harbors. Any ship at any outpost cannot be attacked, nor can any loot on those ships be claimed. And the guns defending the outposts are deadly accurate should you even attempt to breach this accord. Though just because it's a safe harbor while you're in it doesn't mean you'll always be allowed to be safe there. Any player who even attempts to enter a safe harbor while being chased by a ship that is actively hostile towards them, that has shot them within the past 10 minutes and is within 1 map grid's width of you before you enter the harbor, is allowed to chase down their prize, so no running home to mommy to avoid confrontation! If an enemy engages you, you must engage them!
Essentially, the goal of sea of thieves is a mixed PvPvE experience. The game does stand to have a largely PvE focus, as well as having its PvP expanded. The idea is to find ways for people who get stressed out from PvP to have a way to safely play, but also still encourage players to challenge each other, since the game was designed from the ground up with player combat in mind, but the game could be expanded quite heavily if players could interact with each other in less hostile environments when they want to have a less stressful experience. You just want to find ways to encourage these players to cross the fence and have fun the way the game was meant to be played.
Also, can anyone here say they don't want to be an overpowered skeleton ship that attacks players, or be a player who forms a fleet to attack said overpowered skeleton ship? Seriously, I just want the option to become a boss monster right now even without pve servers because the thought from both sides seems very fun if you actually allow players to do this.