Forts are really in the game to offer players a high risk high reward playstyle, as opposed to the low risk low reward that voyages offer. That's why they get the skull cloud, and turn in for about 10 voyages worth of gold/rep.
So, I think giving this (pretty massive amount of) loot to players that didn't actually risk anything to get it would make it feel a little less important and rewarding. You could make it an obscenely low drop rate, but that would just mean almost nobody is ever going to see it, and at that point it isn't even really worth coding.
I understand that you want to add suspense for solo/duo players, but if this was a common enough drop for that to be a normal occurrence, with no risk, it wouldn't feel all that suspenseful.
Lastly, with the way keys are coded, bringing them to the door of a vault that they don't belong to will despawn them, and you'll lose your chance. Plus with how long it can take to sail around the map (over half an hour from corner to corner if you don't have wind), you'd need to know which fort you're going to for this to be anything but boring. But knowing where you're going would really ruin any suspense. But to tell you where you're going, you'd need a voyage. And what if the key changed hands? You could say that the new crew would get the voyage, but that's even more code they would need to change for a really little addition.
TL;DR - It's a cool concept, but in practice I don't think it would add suspense or rewarding play to the game.