Zerotier on OPNsense ignoring links config

Community Support
1

HI All -

New to opnsense and zerotier, trying to replace a manually configured wireguard mesh backed with frr.

I have three nodes, one is ipv6 only, one is dual stack on a single wan, and the third is dual wan, dual stack on one wan and ipv6 only on the second.

I’ve got a problem where the third node binds zerotier to every interface, including all my internal VLANs, and propagates those addresses to the remote sites to try to connect to. Zerotier also ignores the links primary/spare configuration and uses my backup 5G connection as primary towards the second node.

Here’s the config json from opnsense:

{
    "physical": {
        "10.0.0.0/8": {
            "blacklist": true
        },
        "192.168.0.0/16": {
            "blacklist": true
        }
    },
    "settings": {
        "defaultBondingPolicy": "rapid-active-backup",
        "policies": {
            "rapid-active-backup": {
                "basePolicy": "active-backup",
                "linkSelectMethod": "always",
                "failoverInterval": 1000,
                "links": {
                    "lagg0": {
                        "ipvPref": 46,
                        "failoverTo": "vlan0260",
                        "mode": "primary"
                    },
                    "vlan0260": {
                        "ipvPref": 6,
                        "mode": "spare"
                    }
                }
            }
        }
    }
}

And here’s what the ping to a host at the first node looks like from the third node:

64 bytes from 10.15.1.13: icmp_seq=248 ttl=62 time=1115.650 ms
64 bytes from 10.15.1.13: icmp_seq=249 ttl=62 time=218.107 ms
64 bytes from 10.15.1.13: icmp_seq=250 ttl=62 time=26.955 ms
64 bytes from 10.15.1.13: icmp_seq=251 ttl=62 time=68.217 ms
64 bytes from 10.15.1.13: icmp_seq=252 ttl=62 time=17.493 ms
64 bytes from 10.15.1.13: icmp_seq=253 ttl=62 time=53.670 ms
64 bytes from 10.15.1.13: icmp_seq=254 ttl=62 time=396.934 ms

This is very frustrating as, while complex, the wireguard network used the right routes and this link maintained a sub-20 ms RTT. (It’s within the same DOCSIS headend)

Zerotier also seems very slow comparatively - on wireguard I can pull over 900 Mbps from my DC node to my house, zerotier hasn’t exceeded 60 Mbps or so. I’ve verified in “zerotier-cli peers” that all of my connections are direct.

Looking for some guidance - the idea here seems great but it just isn’t working for me.

2

Sounds a lot like this - higher than 1.8.6 the blacklist is ignored.

3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.