Zerotier-one stalls on startup until restarted

Community Support
1

Hi all

I’m wrapping my head around an issue with the ZT 1.10.1 client on Raspbian 10 buster.

I’m bridging two sites and have followed this tutorial:

Bridge your ZeroTier and local network with a RaspberryPi - ZeroTier Knowledge Base - Confluence (atlassian.net)

So far so good, bridges are started properly on both sites and I’m able to ping across them, etc.

The problem I’m having is that the ZT adapter seems to stall immediately after reboot, throwing massive pings and ultimately losing connection.

After restarting the ZT client (sometimes twice) all goes back to normal, so my workaround for the time being is a cron entry to reboot the client 1 minute after startup. It’s dodgy as hell but it’s the only solution I’ve found so far.

Here’s how the ping from site B to a device on site A looks while I restart the gateway on site A:

pi@raspberrypi:~ $ ping [192.168.1.200]

PING [192.168.1.200] ([192.168.1.200]) 56(84) bytes of data.

64 bytes from [192.168.1.200]: icmp_seq=1 ttl=64 time=121 ms

64 bytes from [192.168.1.200]: icmp_seq=2 ttl=64 time=59.0 ms

64 bytes from [192.168.1.200]: icmp_seq=3 ttl=64 time=60.1 ms

64 bytes from [192.168.1.200]: icmp_seq=4 ttl=64 time=62.4 ms

64 bytes from [192.168.1.200]: icmp_seq=5 ttl=64 time=58.9 ms

64 bytes from [192.168.1.200]: icmp_seq=6 ttl=64 time=60.0 ms

64 bytes from [192.168.1.200]: icmp_seq=7 ttl=64 time=58.6 ms

.

.

.

Restart the gateway:

From [192.168.1.227] icmp_seq=30 Destination Host Unreachable

From [192.168.1.227] icmp_seq=31 Destination Host Unreachable

From [192.168.1.227] icmp_seq=32 Destination Host Unreachable

From [192.168.1.227] icmp_seq=33 Destination Host Unreachable

From [192.168.1.227] icmp_seq=34 Destination Host Unreachable

From [192.168.1.227] icmp_seq=35 Destination Host Unreachable

Gateway restarts, a couple of pings ok, then connection stalls:

64 bytes from [192.168.1.200]: icmp_seq=48 ttl=64 time=122 ms

64 bytes from [192.168.1.200]: icmp_seq=49 ttl=64 time=59.6 ms

64 bytes from [192.168.1.200]: icmp_seq=50 ttl=64 time=61.2 ms

64 bytes from [192.168.1.200]: icmp_seq=51 ttl=64 time=62.0 ms

64 bytes from [192.168.1.200]: icmp_seq=52 ttl=64 time=58.8 ms

64 bytes from [192.168.1.200]: icmp_seq=69 ttl=64 time=1848 ms

64 bytes from [192.168.1.200]: icmp_seq=70 ttl=64 time=809 ms

From [192.168.1.227] icmp_seq=76 Destination Host Unreachable

From [192.168.1.227] icmp_seq=77 Destination Host Unreachable

From [192.168.1.227] icmp_seq=78 Destination Host Unreachable

From [192.168.1.227] icmp_seq=79 Destination Host Unreachable

From [192.168.1.227] icmp_seq=80 Destination Host Unreachable

From [192.168.1.227] icmp_seq=81 Destination Host Unreachable

From [192.168.1.227] icmp_seq=82 Destination Host Unreachable

From [192.168.1.227] icmp_seq=83 Destination Host Unreachable

From [192.168.1.227] icmp_seq=84 Destination Host Unreachable

Manually restart ZT client, couple of slow pings, then stalls again, some pings randomly reply:

64 bytes from [192.168.1.200]: icmp_seq=124 ttl=64 time=2369 ms

64 bytes from [192.168.1.200]: icmp_seq=125 ttl=64 time=1330 ms

64 bytes from [192.168.1.200]: icmp_seq=126 ttl=64 time=290 ms

64 bytes from [192.168.1.200]: icmp_seq=142 ttl=64 time=3013 ms

From [192.168.1.227] icmp_seq=154 Destination Host Unreachable

From [192.168.1.227] icmp_seq=155 Destination Host Unreachable

From [192.168.1.227] icmp_seq=156 Destination Host Unreachable

.

.

.

From [192.168.1.227] icmp_seq=180 Destination Host Unreachable

64 bytes from [192.168.1.200]: icmp_seq=181 ttl=64 time=1162 ms

64 bytes from [192.168.1.200]: icmp_seq=182 ttl=64 time=122 ms

64 bytes from [192.168.1.200]: icmp_seq=201 ttl=64 time=348 ms

From [192.168.1.227] icmp_seq=210 Destination Host Unreachable

From [192.168.1.227] icmp_seq=211 Destination Host Unreachable

.

.

.

From [192.168.1.227] icmp_seq=236 Destination Host Unreachable

64 bytes from [192.168.1.200]: icmp_seq=237 ttl=64 time=1468 ms

64 bytes from [192.168.1.200]: icmp_seq=238 ttl=64 time=428 ms

64 bytes from [192.168.1.200]: icmp_seq=256 ttl=64 time=1118 ms

64 bytes from [192.168.1.200]: icmp_seq=257 ttl=64 time=78.9 ms

From [192.168.1.227] icmp_seq=267 Destination Host Unreachable

.

.

.

From [192.168.1.227] icmp_seq=330 Destination Host Unreachable

64 bytes from [192.168.1.200]: icmp_seq=349 ttl=64 time=258 ms

64 bytes from [192.168.1.200]: icmp_seq=350 ttl=64 time=56.8 ms

64 bytes from [192.168.1.200]: icmp_seq=351 ttl=64 time=56.9 ms

64 bytes from [192.168.1.200]: icmp_seq=352 ttl=64 time=56.0 ms

64 bytes from [192.168.1.200]: icmp_seq=353 ttl=64 time=56.6 ms

From [192.168.1.227] icmp_seq=359 Destination Host Unreachable

From [192.168.1.227] icmp_seq=370 Destination Host Unreachable

64 bytes from [192.168.1.200]: icmp_seq=372 ttl=64 time=113 ms

From [192.168.1.227] icmp_seq=400 Destination Host Unreachable

From [192.168.1.227] icmp_seq=408 Destination Host Unreachable

64 bytes from [192.168.1.200]: icmp_seq=427 ttl=64 time=1119 ms

64 bytes from [192.168.1.200]: icmp_seq=428 ttl=64 time=79.6 ms

From [192.168.1.227] icmp_seq=438 Destination Host Unreachable

From [192.168.1.227] icmp_seq=439 Destination Host Unreachable

.

.

.

Another restart of the client, and all is fine since then:

64 bytes from [192.168.1.200]: icmp_seq=453 ttl=64 time=123 ms

64 bytes from [192.168.1.200]: icmp_seq=454 ttl=64 time=59.0 ms

64 bytes from [192.168.1.200]: icmp_seq=455 ttl=64 time=62.1 ms

64 bytes from [192.168.1.200]: icmp_seq=456 ttl=64 time=60.9 ms

64 bytes from [192.168.1.200]: icmp_seq=457 ttl=64 time=58.2 ms

64 bytes from [192.168.1.200]: icmp_seq=458 ttl=64 time=57.3 ms

If anyone has gone through this I’m happy to get any hints.

Cheers!

2

I have now correlated this with the CPU of the zerotier process going beyond 100%. Also when it’s working normally and I perform a speed test with iperf3 the CPU goes 100% and bandwidth is capped at around 43 Mbps. Is there a way to assign it more than one core in raspbian?

3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.