No, I'm not talking about people sinking you and taking your loot, that's part of the game. Let me explain what I mean...
Sea of Thieves is a very PG title, making it a great family game, or so I thought. After seeing me play, my kids took an interest, so I booted up a Galleon for the two of them, and my wife, and we set off to complete the Shores of Gold tall tales. We completed the first two having only one PvP encounter along the way, which ended the way you'd expect with a 9 and a 13 year old trying to crew a Galleon. No big deal. We weren't after the loot we had anyways. We make it to the Wild Rose quest line, and just as we find the medallions, a Brigantine pulls up. A short fight ensued as they boarded us, and of course we lose. Frustrating, but part of the game. Not wanting to restart the quest, I tell the players that I'm playing with my family, and they can have the assorted loot on our ship, and ask them to just leave us the medallions, so we can continue on our way. Big mistake.
As soon as they became aware they were up against children, the trolling began. My kids were exposed to several racial slurs before I figured out how to mute other players. Once I'd done that, they started sending messages via Xbox. All the while, they camped our Galleon until we scuttled it. Then they found us again, and we had to scuttle again. We hopped servers, and somehow they came to our instance, and began to troll again. By that point, we were all too frustrated to continue, and we haven't played since. My children suddenly have no interest in the grand quest they were once excited about.
Sinking ships and stealing their loot is just part of the pirates life. But now that there are SO MANY PvE options (fishing, tall tales, reasons to explore, sunken ruins, Skeleton ships, ect...) I see no reason why the game can't include a PvE mode, and open up it's audience to more casual players who just want to feel like a pirate for a while. Naturally this PvE mode should have penalties, or even completely disable reputation/gold gain, but there's no longer a reason the game should be locked into PvP mode, with hordes of sweaty players who only enjoy ruining the experience of others.