The Island: A ring of small adjoined islands, like chambers of a nautilus, of successive height and breadth. At the edge of the largest is a large rocky cavern with a lightless pool. From that pool danger is said to rise.
The islets are spread just wide enough for a sloop to pass through slowly, perhaps with one or two passages blocked by shallow rocks.
The pool might host the large beast in a different mode of interaction. Or it might host related creatures, smaller and simpler, more regular.
The Beast: A nautiloid or Devonian-like creature with a hard shell. The shell must be gradually cracked and then the exposed tissue attacked for health damage. Some spots are better to hit than others because breaking the wrong spots will loose other appendages for new modes of attack.
The central idea is that this is a dynamic boss with battles proceeding differently each time, depending on how successful pirates are in targeting vulnerable points. If the large beast is the same in the dark pool, it might be that one encounter (at the island or at sea) leads to the other.
