X doesn't mark the spot???

This is, beyond a doubt, the most infuriating Vault quest I've ever been on. I've NEVER struggled with a treasure map the way I have with this one ! It showed the X on Shark Bait Cove on the Northwest edge of the island, right above a solitary rock, but when I got there, I found that there were in actuality THREE rocks on the beach, so I used my logic, and looked at small landmarks on the isle itself to gauge better where to dig (not my first rodeo, after all, I'm almost to Lv 50 with the Gold Hoarders). I dug, and dug and dug and dug and dug and and DUG and found absolutely NOTHING on that entire side of the island. I double and triple checked that I was on the right island, using my compass and going back to my sloop, and all the other landmarks on the map ECXEPT the rocks all made sense. I eventually had to just quit, because I was getting tired of fending off gunpowder skellies every 3 minutes with nothing to show for my time. I know it should have been there, and I usually only have to dig 1-3 times before striking my treasure when using maps, so this was extra infuriating. Any ideas as to what I was doing wrong? I wish I could provide screenshots of the map...