By default Windows 10 is configured incorrectly and does not allow problem free multiplayer or party chat.
Here is how you can check if you are affected:
Open the Xbox App, click on the little gear symbol at the bottom to open the settings tab and finally click network.
It's very important that the NAT type is open and not moderate or even strict.
Microsoft explains the NAT types as follows:
So only if you have an open NAT you can play with truly everyone else without running into issues.
To fix your NAT type follow these instructions: https://support.xbox.com/en-US/xbox-on-windows/social/troubleshoot-multiplayer-game
Especially the part about setting a static Teredo port because by default Windows 10 does not have it set to static and you won't get an open NAT type.
So simply right-click the Start icon, and then select Command Prompt (Admin) and run the following command from the Admin command prompt:
netsh interface Teredo set state client clientport=60209
Then check again in the Xbox app if your NAT type is now set to open.
Also be sure to forward the ports in your router as described in the Microsoft link, and also double check that you did not forget to forward the 60209 port.
You might need to reboot your PC and router though to see any effect.
I hope this helps people with chat or connection issues on Windows 10.