I see a lot of times people complaining about their Tall Tales items being stolen and possible ways to tell other crews your loot is useless for them.
I speak for myself and cannot tell if other crews might do the same, but you can always try to speak to the other crew and tell them you are doing a Tale.
I know this can sound weird but it might work. There has been two times that when I was attacking a ship and someone told they were doing a tale. In those two occasions my crew and I decided to let them sail but stole all the other stuff they had on board.
With this I want to encourage you to speak because you will never know how useful this is until you try it. Of course, not all the pirates are like me and will let you keep your Tale items or even give them back to you, but you can always try and ask them.
I hope this little tip may help you. This community has really great people playing and many of them may understand the struggle of doing Tall Tales and could let you sail to finish it.