Xbox one controller disconnects

I am just now getting back into SoT after a year but it’s hard currently because when I try to play my controller is disconnecting constantly. The weird thing is that it only disconnects when playing sea of thieves. Any other game or app on the Xbox works fine. But for some reason sea of thieves makes the controller disconnect at least every 2 minutes which is frustrating. Is this a common problem?