I’m posting this in the vain hope that someone is also having these issues. After upgrading zerotier-one to 1.6.4, all of my peers stopped responding to each other. The peers show up as ONLINE in my.zerotier.com as well as in zerotier-cli, but pinging from any host to any other yields “Destination Host Unreachable”. I’m testing mainly from Linux peers, but the Windows peers are also not responding.
I’ve tried downgrading the app to 1.6.2, but it didn’t help.
zerotier-cli listnetworks
zerotier-cli peers
For listnetworks it should show your current network and the status on the right side, if this is OK check the next.
For peers it should list all of the “other” devices on the ZT network, if they ALL say “LEAF” “RELAY” you are probably behind a double NAT without port forwarding, try forwarding UDP port 9993, it always does the trick for me.
Everything is in order: networks say OK PRIVATE, all the leaves are DIRECT.
The peers that I’m testing are listed on each other using their IPv6 addresses, but that didn’t stop them from working before.
There were no network changes. I’ve been using Zerotier for a few years now and it’s the first time I cannot seem to do anything to fix this.
That rules out almost everything.
Re-check your network rules, i’d recommend setting the defaults.
Also is your default route in there?
Say your clients are using 192.168.50.x
make sure there is a route for that subnet ex. 192.168.50.0/24, if you are not sending the traffic to any other gateway leave the box blank for “VIA”, it should say (lan) when added. (if you are sending it to another gateway it gets significantly more complicated)
I have only one route in the ZT panel: 10.x.y.0/21(LAN) and it’s not segmented anywhere at the moment (all peers are just endpoints).
However I have dusted off another endpoint (my RaspberryPi 4), and found out that I can connect it to and from my PC (on the same LAN). Also another Windows endpoint (external to my LAN) started to work from my PC.
I narrowed it down to just one instance not working - an OpenVZ VPS, which coincidentally I’ve been using as an internal DNS server. The connections stoppped working there and there’s a weird symptom when pinging any ZT host from there:
# ping 10.x.y.2
PING 10.x.y.2 (10.x.y.2) 56(84) bytes of data.
From 10.x.y.1 icmp_seq=2 Destination Host Unreachable
From 10.x.y.1 icmp_seq=3 Destination Host Unreachable
From 10.x.y.1 icmp_seq=4 Destination Host Unreachable
ping: sendmsg: No buffer space available
ping: sendmsg: No buffer space available
ping: sendmsg: No buffer space available
ping: sendmsg: No buffer space available
^C
--- 10.x.y.2 ping statistics ---
12 packets transmitted, 0 received, +3 errors, 100% packet loss, time 38026ms
pipe 3
I tried downgrading the zerotier version there to no avail. I just may have to ditch that VPS…
weird…
what is the ZeroTier CPU and Memory usage on that VM?
I got rid of that VPS via support, and they provided me a new one on another node with different OpenVZ version.
Unfortunately, I can’t remember the exact usage of zerotier service on there but “free” command showed about 80 MB used memory out of 128MB available.
However, now I have a different issue - I’m getting PORT_ERROR when trying to join a network, even though I’ve enabled TUN/TAP support in the VPS control panel.
There’s an error from zerotier-one service:
ERROR: unable to configure virtual network port: could not open TUN/TAP device: No such file or directory
I’m honestly tired…
EDIT: I found out this issue: https://github.com/zerotier/ZeroTierOne/issues/699
The permissions to /dev/net/tun are correct, but the /dev/net folder aren’t:
host [/dev/net] # ls -al
total 0
drwx------ 2 root root 60 Feb 24 01:01 .
drwxr-xr-x 9 root root 1280 Feb 24 01:01 ..
crw-rw-rw- 1 root root 10, 200 Feb 24 01:01 tun
I managed to avoid this issue by setting -U parameter in zerotier.service