Frazzled with managed routes: Road v In-Office

Community Support
1

I wanted to set up a route to our office LAN to replace the OpenVPN system we are using. This seemed easy enough. I didn’t even mess with trying to create a real bridge, I just set up a node in the office and created routing rules on the appropriate firewalls/routers to direct traffic back and forth. Everything works great from my house.

In the office, though, it created some chaos. The route being pushed by ZeroTier overrode the existing routes to the LAN and resulted in systems that couldn’t use the network until a reboot.

I read that playing with netmasks may solve this, so instead of routing 10.151.0.0/16 over the ZeroTier gateway node, using 10.151.0.0/15 instead. And at first this seemed to work, but in the end it isn’t working well either.

Even stranger, and all our user systems are MacOS 10.14 Mojave, when they first boot and join the ZeroTier network, the managed routes are not being pushed down or implemented at all. That is why I thought the netmask thing was working, because they rebooted and kept internet access, but if they leave the ZeroTier network and re-join it, the routes get all messed up.

On reboot:
default is 10.151.0.1
10.151/16 is on en0

After I leave and re-join the network:
default is the ZeroTier gateway node ZT IP addr
10.150/15 is the ZT gateway node ZT IP addr
10.151/16 is en0

I don’t want the “default” route to be over the ZT network. And the 10.150/15 is how it shows up when I push the 10.151.0.0/15 route so as to not stomp on the 10.151.0.0/16 LAN route.

How can you get these to co-exist so you can have an “always-on” ZT path to your LAN and yet not mess up the LAN routing when you are in the building? Surely this has been done. :slight_smile: Or do I have to move to some other approach like L2 bridging? But I rather like keeping the subnet separate so I can apply some filter rules on ZT traffic.

To be abundantly clear, the route I put in the ZeroTier web UI to push was 10.151.0.0/15 via the ZT gateway node ZT IP addr.

Help. :slight_smile:

2

Hello. sorry this took up a bunch of time.
Try 10.150.0.0/15 for the zerotier managed route. Something somewhere in the chain is particular about that.

    # ipcalc 10.151.0.0/15                                                                                                                                                  ⏎ release
    Address:   10.151.0.0           00001010.1001011 1.00000000.00000000
    Netmask:   255.254.0.0 = 15     11111111.1111111 0.00000000.00000000
    Wildcard:  0.1.255.255          00000000.0000000 1.11111111.11111111
    =>
    Network:   10.150.0.0/15        00001010.1001011 0.00000000.00000000
    HostMin:   10.150.0.1           00001010.1001011 0.00000000.00000001
    HostMax:   10.151.255.254       00001010.1001011 1.11111111.11111110
    Broadcast: 10.151.255.255       00001010.1001011 1.11111111.11111111
    Hosts/Net: 131070                Class A, Private Internet
3

I did, that was my point, it didn’t prevent this chaos.

4

It’s 150, not 151?

5

Ugh, see how frazzled I am? The actual net is 10.151.0.0/16. The route I put in ZT GUI was 10.151.0.0/15, but when I display the route using the tools in macOS they show up as 10.150.0.0/15. Why? I don’t know. But the bottom line is I can’t get them to coexist, and even worse, the behavior of ZT is different on a clean boot and auto-join of the network than it is if I leave and re-join the network.

So you are saying to actually push 10.150/15? Hmmm… I’ll have to try that, but it is hard remote because when things go south I lose the ability to get to the systems at all and have to drive into the building to reboot them.

Need to figure out, though, why the initial boot/config of ZT doesn’t get the routes pushed to it at all, but a leave/join of the network does. That is separate from the netmask issue, but it will be important to resolve as otherwise things aren’t going to work as intended. Right now I don’t have the ZT GUI loaded on the client systems as I don’t want them disconnecting from the network, so they can’t just join/leave to get the routes right, and rebooting doesn’t help because they don’t get added.

6

Also, and this may not be fixable, but there two other subnets connected to the LAN that I want to have remote access to… 192.168.3.0 and 192.168.15.0. The netmask trick won’t work for those because when on the LAN there is no specific route pushed, they just work because the default gateway knows how to get to them. So that is going to require some tool to add/remove those routes on demand, and that point, I might as well just do all of them that way. Not ZT’s fault on those other subnets, and I don’t want to try to push manual routes on all the systems all the time because when they aren’t in the building it may cause other issues. {sigh}. So much fun, but still, ZT offers a lot of promise, just need to get it working for us. Probably slated to go the paid tier if I can get management to agree on the value.

7

This is to be expected. In CIDR parlance, there is no 10.151.0.0/15. Only 10.150.0.0/15. It encompasses 10.150.0.0 through 10.151.255.255.

10.151.0.0/16 is a valid CIDR block, but when you want to expand it to a /15 it becomes 10.150.0.0/15. That’s the result of:

11111111.11111110.00000000.00000000 (the /15, aka 255.254.0.0)
& 
00001010.10010111.00000000.00000000 (10.151.0.0)
=
00001010.10010110.00000000.00000000 (10.150.0.0)
8

You can also add the following routes in your Managed Routes on your ZeroTier:

192.168.3.0/24 via $GATEWAY_IP_ADDRESS
192.168.15.0/24 via $GATEWAY_IP_ADDRESS

It won’t hurt anything

9

@zt-grant Thanks! Is that documented anywhere that I missed? I’ll give it a shot after-hours, I don’t want to break things again during the work day. :slight_smile:

And do you mean to actually put ‘$GATEWAY_IP_ADDRESS’ or to put the actual address of the gateway network? I’m thinking maybe the later, and then it makes more sense as the path would be the same whether on ZT or on the LAN. Thanks for the clue on that one. That would only break down if the ZT node somehow didn’t get the route set to the LAN segment, like we are seeing on reboots, but I’ll play with it.

Actually, I tried, 192.168.3.0/24 via 10.151.0.1, but I get network unreachable.

10

I put $GATEWAY_IP_ADDRESS as a variable because I don’t know your local lan configuration.

And I’m not sure what you’re referring to when asking if something is documented?

11

As for “is it documented”, I meant the use of a variable when creating the route. :slight_smile: But you didn’t mean it literally.

So, I have the route 10.150/15 installed as previously suggested so it didn’t conflict for LAN users, and if I add the 192.168.15.3 via 10.151.0.1 (the LAN gateway to that subnet), I get network unreachable while on ZT.

If I put in the ZT gateway IP of the host on the LAN that is the ZT host, it works for me, but then it would push that route to the LAN which isn’t what they should use.

Just to clarify, after doing this, I have the following routes:
10.150/15 via ZT GW HOST IP on ZT face
192.168.3.0/24 via 10.151.0.1 on ZT iface

10.150/15          172.22.120.113     UGSc       feth466       
192.168.3          10.151.0.1         UGSc       feth466

I can ping 10.151.0.1, but pinging anything on 192.168.3.0/24 gives me the network is unreachable.

On macOS I can do a ‘route get 192.168.3.2’ and see:

   route to: 192.168.3.2
destination: 192.168.3.0
       mask: 255.255.255.0
    gateway: 10.151.0.1
  interface: feth4668
      flags: <UP,GATEWAY,DONE,STATIC,PRCLONING>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0
12

Hrmm… Possible that you’ll just have to stick with 192.168.3.0/24 via $ZT_GATEWAY_HOST. Yes it means that when the computers are on the local LAN they will route via the ZT gateway host. While not ideal, it shouldn’t be bad either.

Only other solution I can think of is to explicitly push the routes to those subnets via DHCP on the local LAN. then you can set 192.168.2.0/23 via $ZT_GATEWAY_HOST (same basic binary math as above for why 10.151.0.0 goes to 10.150.0.0 when the mask is changed from /16 to /15)

13

I’ll play with it, but thanks very much for trying to help.It seemed that when the route was pushed with the ZT GW IP it didn’t work, but that may have been when the LAN was done that way, not an extra subnet. I really appreciate your input.

1 Like
14

Just for the record, here is how I had to configure to push the static routes via DHCP so ZeroTier overrides didn’t overtake them. Previously the default gateway just handled the routing to these subnets. However, I had to take this out at the moment because some systems on our network did get messed up after I implemented this, I don’t know why. But still I thought it might be worth sharing how this was set up for ISC DHCP:

# Push static routes
option classless-routes code 121 = array of unsigned integer 8;
option classless-routes 00, 10, 151, 0, 1, 24, 192, 168, 15, 10, 151, 0, 1, 24, 192, 168, 3, 10, 151, 0, 1 ;
option classless-routes-win code 249 = array of unsigned integer 8;
option classless-routes-win 00, 10, 151, 0, 1, 24, 192, 168, 15, 10, 151, 0, 1, 24, 192, 168, 3, 10, 151, 0, 1 ;

The above routes default to 10.151.0.1, maps 192.168.15.0/24 to 10.151.0.1 and 192.168.3.0/24 to 10.151.0.1. It uses both option 121 and 249 for Windows.

The format is subnet mask, IP, GWIP. Not that all ‘trailing’ 0 segments are left out, so 0.0.0.0 has no entry, and 192.168.3.0 is just 192.168.3.

Maybe someone would find this useful at some point down the road. :slight_smile:

1 Like
15

This topic was automatically closed after 14 days. New replies are no longer allowed.