[Guide] Setup ZeroTier to connect 3 sites with IPv4 and IPv6

Tips and Guides
, ,
1

I manage networking on 3 locations. Every location has a LAN with IPv4 and IPv6. I want to connect these LAN’s with ZeroTier. Various websites and articles did give me some pointers to get this working. Maybe some people find my setup helpful.

ZeroTier is installed on an EdgeMAX router (see https://kruyt.org/zerotier-on-a-ubiquiti-edgerouter how to install) on Site1 & 2, Site3 has ZeroTier on an OPNsense router (see https://docs.opnsense.org/manual/how-tos/zerotier.html)

ZeroTier network setup

The controller is installed on a AWS EC2 instance, port 9993/UDP has to be open to the internet. Configuration is done with some API commands.

Get the authorisation secret and put it in a variable:

TOKEN=$(sudo cat /var/lib/zerotier-one/authtoken.secret)

Get your nodeid and put it in a variable:

curl -X GET "http://localhost:9993/status" -H "X-ZT1-AUTH: ${TOKEN}"

The output will show your nodeid in “address”.

NODEID=8d6xxxxxx8

Create your network and put it in a variable:

curl -X POST "http://localhost:9993/controller/network/${NODEID}______" -H "X-ZT1-AUTH: ${TOKEN}" -d {}

Posting ${NODEID}______ will generate a random networkid.

NWID=8d6xxxxxxxxxxx65

Configure the network:

curl -X POST "http://localhost:9993/controller/network/${NWID}" -H "X-ZT1-AUTH: ${TOKEN}" \
  -d '{"ipAssignmentPools": [{"ipRangeStart": "172.20.192.1", "ipRangeEnd": "172.20.192.1"}], \
  "v4AssignMode": "zt", "private": true, "v6AssignMode": { "6plane": false, "rfc4193": true, "zt": false } }'

Router setup

On the EdgeMAX router confuguration can be done via the CLI.

sudo zerotier-cli join 8d6xxxxxxxxxxx65

Allow routing configuration of global IPv6 adresses on all clients:

sudo zerotier-cli set 8d6xxxxxxxxxxx65 allowGlobal=1

Get the nodeid:

sudo zerotier-cli info

On OPNsense configuration is done via the GUI, make also a note of nodeid.

Authorize the routers on the ZeroTier controler

Every router should be authorized with his nodeid:

curl -X POST "http://localhost:9993/controller/network/${NWID}/member/48xxxxxxe9" -H "X-ZT1-AUTH: ${TOKEN}" -d '{"authorized": true}'

After authorisation make a note of the ip-addresses given to the routers.

ZeroTier network routing

Site1:
network: 10.70.10.0/24, 2001:41xx:xxxx:xxxx::/64
router: 172.20.192.1/24, fd8d:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:1ae9/88

Site2:
network: 10.80.20.0/24, 2001:1cxx:xxxx:xxxx::/64
router: 172.20.192.2/24, fd8d:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:2393/88

Site3:
network: 10.90.30.0/24, 2aof:29xx:xxxx:xxxx::/64
router: 172.20.192.3/24, fd8d:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:2c98/88

Configure the routing:

curl -X POST http://localhost:9993/controller/network/${NWID}/ -H "X-ZT1-AUTH: ${TOKEN}" \
  -d '{ "routes": [{"target": "172.20.192.0/24", "via": null}, \
  {"target": "10.70.10.0/24", "via": "172.20.192.1"}, \
  {"target": "10.70.20.0/24", "via": "172.20.192.2"}, \
  {"target": "10.70.30.0/24", "via": "172.20.192.3"}, \
  {"target": "fd8d:xxxx:xxxx:xxxx:xxxx::/80", "via": null}, \
  {"target": "2001:41xx:xxxx:xxxx::/64", "via": "fd8d:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:1ae9"},
  {"target": "2001:1cxx:xxxx:xxxx::/64", "via": "fd8d:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:2393"}, \
  {"target": "2aof:29xx:xxxx:xxxx::/64", "via": "fd8d:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:2c98"} ] }'

With the right local DNS service, all devices on all sites can now reach each other.

1 Like