Using ZeroTier to create a "Site to Site" connection

General Discussion
1

Hi guys
I am new on Zerotier so I need a little help on how to setup a “Site to Site” connection.

Office 1 - 192.168.3.0/24
#OPNSense Firewall/Router 192.168.3.1/24

Office 2 - 192.168.2.0/24
#OPNSense Firewall/Router 192.168.2.1/24

On each site I have installed the ZeroTier app and joined then network.
I need have that every client on Office1 is able to PING and connect any client or resources on Office 2 by using the internal network IP and vice-versa.

There is any guide that I can follow or maybe some one can help please?

Thanks

2

Hello,
You can add Managed Routes to the my.zerotier.com network settings to push routes to your physical subnets.
(Your zerotier subnet should NOT overlap the physical subnets)

Managed Routes should look something like
10.147.20.0/24 (zerotier subnet)
192.168.2.0/24 via 10.147.20.2
192.168.3.0/24 via 10.147.20.3

I don’t know about the specifics of configuring opnsense.

3

Thanks for your replay
I will try your suggestion and let you know.

Ciao!

4

Unfortunately is not working…
I can’t ping any of the host on Site2 from Site1 and vice versa…

:frowning_face: :frowning_face: :frowning_face: :frowning_face:

5

what happens when you traceroute to the other site? opensense might be blocking at a firewall level… its been a while since i used it, so cant help either, but check opensense logs…

6

In the mean time I did some more testing and I want share these with you.

I installed ZT on my laptop and connect it to the ZT network via 4G.

So now I have the follow:

Office 1 - 192.168.3.0/24
#OPNSense Firewall/Router [192.168.3.1/24] (ZeroTier static IP 192.168.193.3)

No firewall rules add to OPNSense

Office 2 - 192.168.2.0/24
#OPNSense Firewall/Router [192.168.2.1/24] (ZeroTier static IP [192.168.193.2]

No firewall rules add to OPNSense

Laptop - 4G connection

#(ZeroTier static IP [192.168.193.30]

Right now I can do the follow:

Laptop -> can ping Office1 and Office2 clients by using the internal private IP [192.168.2.0/24] & [192.168.3.0/24]

Office 1 -> can ping Laptop on ZT IP but can’t ping Office 2

Office 2 -> can ping Laptop on ZT IP but can’t ping Office 1

What I should do now?

7

My guess is you need to add the routes to your opnsense routers.

8

You probably right but I don’t know how!
Anyone with OPNSense skills that can help me?

Thanks

9

I’m having the same problem - is there anyone out there who can help, please.
Thanks JJ

10

Hello,
You can add Managed Routes to the my.zerotier.com network settings to push routes to your physical subnets.
(Your zerotier subnet should NOT overlap the physical subnets)

Managed Routes should look something like
192.168.193.0/24 (zerotier subnet)
192.168.3.0/24 via 192.168.193.3
192.168.2.0/24 via 192.168.193.2

open the zerotier control panel and click on “Allow Default Route Override”,both office1 and office2

11

Hey Gerry
I did but still no working
Not sure what else can be

12

We have done this. Not with OPNSense, but with Ubiquiti routers.

The process is relatively simple - even for a device which isn’t by rights designed to do it.

To add the static route on OPNSense, follow this:- Routing — Static Routes | pfSense Documentation

As gerry said, you need to add the routes to my.zerotier.com first - then add the routes in to OPNSense.

13

I have the same problem here and tried all sorts of different ways to solve this. It does seem there is some ZT network traffic originating from my internal interfaces, local lan interfaces, that doesn’t make it to the destination peer. Is there a way to bind ZT to a specific interface and have ALL traffic originate from there?

15

Hello guys,

I just want to share what i’ve been successfully done atfer
struggling couple days to solve it.

I have 3 opnsense server node at 3 different cities.
Each of them are using Internet Service provider with private IP
a.k.a “behind CGNAT” (no Public ip address at all).
All of those 3 opnsense server using >> “zerotier plugin” <<
to connect to zerotier central service.

For furter Reference, i did setup each of opnsense server base one
tutorial on youtube: https://www.youtube.com/live/Zp5vKPLAYdc?feature=shared

I did setup each of them as follows:

Opensense City 1 (Jakarta) – Zerotier Assigned IP: 10.144.77.1
in this server i have two LAN Subnet:
– Main Office: 192.168.1.0/24
– Family: 192.168.2.0/24
– Servers Farm:: 192.168.3.0/24

Opensense City 2 (Bali) – Zerotier Assigned IP: 10.144.77.2
In this server i have three LAN Subnet:
– Motel Room: 10.10.0.0/16
– Bar-Resto-Fitnes: 10.20.0.0/16
– Management Office: 10.30.0.0/24

Opensense City 3 (San Diego-CA) – Zerotier Assigned IP:10.144.77.3
In this server I have two LAN Subnet:
– Family: 192.168.99.0/24
– Servers Farm: 192.168.88.0/24

Afther finished doing that tutorial i do have same problem with him,
sometimes connected couple minutes then droped then connect again.
Or sometimes it doesn’t connected at all all day long. Also it have spikes
of the “Zerotier packets” on each of the server - i assumed it was
called >> “software laser issue” <<, to solve it i do couple things as follow:

At Zerotier Central web Application: >> Advanced>Managed-Routes << i put following
route configuration as follows:

192.168.1.0/24 via 10.144.77.1
192.168.2.0/24 via 10.144.77.1
192.168.3.0/24 via 10.144.77.1

10.10.0.0/16 via 10.144.77.2
10.20.0.0/16 via 10.144.77.2
10.30.0.0/24 via 10.144.77.2

192.168.99.0/24 via 10.144.77.3
192.168.88.0/24 Via 10.144.77.3

Then I put “local.conf” configuration code on every opnsense server
(Jakarta, Bali, San Diego-CA). This can be done via opnsense web gui administration
which is at >> “VPN:Zerotier:Settings” << as follows:

{
“physical”: {
“192.168.1.0/24”: {
“blacklist”: true
},
“192.168.2.0/24”: {
“blacklist”: true
},
“192.168.3.0/24”: {
“blacklist”: true
},
“10.10.0.0/16”: {
“blacklist”: true
},
“10.20.0.0/16”: {
“blacklist”: true
},
“10.30.0.0/24”: {
“blacklist”: true
},
“192.168.99.0/24”: {
“blacklist”: true
},
“192.168.88.0/24”: {
“blacklist”: true
}
}
}

Save & Apply !

It requires to restart every single opnsense server above
(Jakarta, Bali, and San Diego-CA) to work properly.

In result,
any of PC computer/laptop/phone (whithout installing zerotier on PCs/laptop/phone) that connected
to LAN network on one city could connect to any PC Computer/server on the two others cities
and vice versa. For example, i have laptop connected to LAN on opnsense server at jakarta
that have ip address: 192.168.1.7, it can connect file sharing on the NAS Server
which sitting on the opnsense at San Diego-CA, by simply connect to NAS Server local ip address 192.168.88.8.

If you would like to limit it, - based on your needs - you have to configure 2 things as follow:

  1. Configure >> “Advanced>Managed Routes” << on Zerotier central web Application.
  2. Configure firewalls rules at >> “Firewall:Rules:Ztier” << on every single opnsense server
    connected to zerotier central.

I hope this can help others who have same difficulty to solve.

Regards,
Mukky Van Djava.