Absent any improvements to the netcode, the solution to me is clear, even if it seems obvious and has been done before in other games.
Create new instanced areas using the portals that limit the number of ships to 2, maybe 3.
The hunters cry was the best example of what this mode could offer in which multiple crews could be merged together into a new instanced area for a more tailored experience. Sure, some elements of the netcode could still suffer (like hitreg), but at least any new content / biomes would not dilute the regular adventure mode further. This also has the advantage of still allowing xbox one support by letting Rare instance any new content, which should, hopefully allow for higher quality content without the performance hit of putting it all in the main server.
As a roadmap for how I imagine these "dungeons" should be implemented:
1 - Allow some "dungeons" to be have the choice to be solo only (for a lesser reward), or invadable for a greater reward. Overall though, the goal should be to create more challenging PvE encounters, limit supplies, and randomize some objectives for that "next level" reward (i.e. only bananas allowed or complete the event with only 10 planks). Rewards wouldn't be loot your bring back with you, but it could be a combination of rewards you receive at the end that scales with the amount of loot you gather during the voyage. Gold would be the default consolation prize for a satisfactory completion, but the real treasure would be cosmetics, trophies, etc. for meeting the objectives.
2 - Requires some combination of items to initiate at a ritual sites at some nearby island (example: athena chest + ashen winds = Ashen Athena Dungeon, or castaway chest + chest of rage = PvP dungeon). Make some combinations consistent (result in a specific dungeon with specific known victory conditions), but make other combinations random for each server (maybe you discover a secret dungeon by combining a ritual skull + a banana crate but only sometimes). Possibly, have some items that persist between sessions which are tied to your character that you gather over time, and can choose to use later. Some of these items your receive by doing lower level dungeons which you then parlay to do higher level dungeons. Rare can monetize by giving players a means to shortcut this by paying some ancient coins to bypass the need for these items.
Find clues in barrels that can tell you some specific combinations that are needed for that server, and gain some maps to specific items needed (castaway chests, etc) using the existing message in bottle system. This rewards crews for staying in the same server (more clues & opportunity to gather necessary ritual items), and for teaming up with other crews to uncover which dungeons this server is featuring. Maybe you need to use certain ship cosmetics as well, for example, and can spot other crews looking to initiate the same dungeon based on their ship appearance (have merchant sails or athena sails, etc). If server differences are too difficult, the requirements could change each day / week at a set time.
3 - Make some events mandatory to have more than one crew, with a mix of only cooperative (PvP and interact on other ships disabled), or competitive (crews engage in a tug of war with some PvE sprinkled in). Some specifics will need to be worked out on how exactly to queue ships together, but I think crews should be given two options. Either wait for a crew in the same server to also offer the same items or "start matchmaking." This gives crews the flexibility to ask around and see if other crews in the server want to take on a dungeon together. Competitive dungeons though I think should come with some caveats. Maybe still allow crews in the same server to queue up but only allow each ship to do this once per session (tied to boat not to players) so that it is not abused with alliance servers trading victories. If you want to do multiple competitive dungeons per server, you will need to matchmake.
Note that once a ship is merged into an instanced dungeon, this ship is now off server so it won't be returning to the same server. This likely means you won't be able to take on multiple of the secret dungeons since the combination will be different once you are merged back into the main sea of thieves.
I think overall, this experience is what is missing from Sea of Thieves. PvE only is oft requested and this is one way to fulfill that need without diluting what makes the sea of thieves great. In addition, finding a way to create a cyclic economy between what we farm in adventure mode (gold and doubloons) and what we might earn in the dungeons (trophies, titles, cosmetics, maybe even coupons to the emporium / ancient coins) is a sound strategy to give players a reason to play. Of course, Rare would need to find a way to monetize this, but I think offering something like this is a very good way to retain players, get new players hooked, and just keep the game alive. That should be reason enough as there are a lot of cosmetics they can monetize already, and the real prize for them is getting more players into the game.