I’ve recently upgraded ZT to 1.8.4, and am noticing a problem with the way ZeroTier adds static routes on linux (CentOS 7). The routes are added upon startup of the zerotier-one daemon as expected, but they appear to be getting added before the ZT interface (zt44xi7gkv) is up in some cases. In my network I have three static routes corresponding to the ranges in rfc1918. Sometimes the routes get added via the zt44xi7gkv interface in which case traffic flows as expected. Other times they get added using my VM’s interface, venet0, in which case traffic to those ranges is not reachable. In these cases the zt44xi7gkv interface is still up, but I suspect it came up after the routes were already added. If I restart the zerotier-one daemon, some, or even all of them might get added correctly, or not. Is this a known issue?
Non-working case:
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 0.0.0.0 0.0.0.0 U 0 0 0 venet0
10.0.0.0 100.64.10.47 255.0.0.0 UG 0 0 0 **venet0**
100.64.10.0 0.0.0.0 255.255.255.0 U 0 0 0 zt44xi7gkv
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 venet0
172.16.0.0 100.64.10.47 255.240.0.0 UG 0 0 0 **venet0**
192.168.0.0 100.64.10.47 255.255.0.0 UG 0 0 0 **venet0**
Semi-working case: (172.16.0.0, 192.168.0.0 correct, 10.0.0.0 not correct)
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 0.0.0.0 0.0.0.0 U 0 0 0 venet0
10.0.0.0 100.64.10.47 255.0.0.0 UG 0 0 0 **venet0**
100.64.10.0 0.0.0.0 255.255.255.0 U 0 0 0 zt44xi7gkv
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 venet0
172.16.0.0 100.64.10.47 255.240.0.0 UG 0 0 0 zt44xi7gkv
192.168.0.0 100.64.10.47 255.255.0.0 UG 0 0 0 zt44xi7gkv
Expected result:
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 0.0.0.0 0.0.0.0 U 0 0 0 venet0
10.0.0.0 100.64.10.47 255.0.0.0 UG 0 0 0 zt44xi7gkv
100.64.10.0 0.0.0.0 255.255.255.0 U 0 0 0 zt44xi7gkv
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 venet0
172.16.0.0 100.64.10.47 255.240.0.0 UG 0 0 0 zt44xi7gkv
192.168.0.0 100.64.10.47 255.255.0.0 UG 0 0 0 zt44xi7gkv