Slow/Unstable TCP over Zerotier on T-Mobile Home internet (5G)

Community Support
1

I use ZeroTier at the company I work with to provide our VPN layer so people can access our sites/office remotely (we do pay for the web controller service). I’ve got a user who is having major trouble getting TCP connections to work over his new T-Mobile 5G home internet. We’ve tried iperf3 tests between two different Zerotier Endpoints and gotten similiar results. It seems the TCP connection gets formed but starts to fail immediately (lots of retries and ultimately it either fails or gets up to a small bit of bandwidth that is unstable). UDP Stream to/from these endpoints do not experience any issues - no packet loss, no bandwidth issues. I’ve tried large/small UDP packets to shake things up and it is the same, only TCP over the tunnel is a problem.

I’ve had the user try a Verizon Hotspot, and they have no issues with it. TCP is fine on that connection over zerotier. I’ve also set up an exposed iperf3 server at one of the remote ZT endpoints and tested TCP streams direct to it (not through Zerotier) and that had no issues.

Unfortunately this is not a strictly ZeroTier problem. I set up a WireGuard tunnel the other day between one of the endpoints and this user and we experienced roughly the same result (slightly better TCP performance/stability, but still not good enough to do anything).

I suspect T-Mobile is shaping UDP (maybe only “unrecognizable” UDP traffic?) traffic somehow in a way that is getting in a feedback loop with TCP’s own shaping algorithm causing a sort of melt-down. Since UDP does not shape itself, it’s getting through OK.

Unfortunately I don’t have a good way to see the “health” of the Zerotier (or even wireguard for that matter) connection to know if it is being shaped at all, or in what ways. I am also dreading having to go through T-Mobile support to try and fix this issue (I am doubtful they can/will for a residential line).

Has anyone any wisdom here besides “get a different ISP”? Or any insight into the kind of shaping a mobile ISP even does that could cause a “TCP Only” failure like this inside an encrypted tunnel? I am not versed in the mysterious ways of residential ISPs.

2

That’s a tricky one and unfortunately you have no control over the physical network settings.
Do you know if the 5G gives you IPv6 or v4 or both? If it’s v6, does the other node have v6 internet access as well?

3

It probably does - I will have to check that and set up a test scenario, our main office does not have IPv6 routed through the network so I will have to test some other location/segment. I’ll report back if the direct connection (No NAT/Ipv6) makes a difference.

4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.