The seabound soul bug?

I'm on the second part of the tall tale, where you have to find the second skull. The problem I am having is, when I go down and use the lantern nothing show up and yes, I am using the correct lantern. This second ship, in my opinion is too far away from the island. I've swam all around, drowned several times while trying to find it. This is the second time I've tried this and the same thing has happened both times. Is anyone else having this issue?