I am not sure if this is intended, but there is a way to lose one of your Ashen Keys on the key-seeker voyage before you even obtain it.
So with the Key Seeker Voyage, you dig up the first key. Upon digging, another Ashen Lord spawns as an ambient skeleton encounter.
If you are solo and die, or if your entire crew dies, the island will reset and thus de-spawn the ambient skeletons that spawn upon digging up a treasure. This has always been in the game.
But I noticed this mechanic had an adverse effect on our ashen key seeker voyage by despawning the Ashen Lord holding the second key, making the second key unobtainable since the second Ashen Lord was technically an ambient skeleton that spawns upon digging up the first key.