Getting mixed results on a not-so-simple setup

Community Support
1

Hi!

I’m new to the community, bu I’ve been trying out ZT for a couple of months already.

Here’s my setup:

Client #1 (on-premise):

  • Running as a VM (Host is running LinuxMint 20/Ubuntu 20.04)
  • OPNsense with ZT 1.6.5
  • Ethernet port used for WAN interface is dedicated to VM (passthrough)

Client #2 (cloud-hosted):

  • Running as a VM (Host is a bare metal instance running Ubuntu 16.04)
  • OPNsense with ZT 1.6.5
  • UDP port 9993 is forwarded by host to VM
  • WAN ip address is on a private subnet: 192.168.x.y/24
  • Gateway is the ip address of the host on the 192.168.x.y/24 subnet

Client #3 (on-premise):

  • My workstation
  • Running Ubuntu 20.04 and ZT 1.6.5
  • No ports are forwarded to my workstation from the firewall/router.

Client #4 (on-premise, another/different site):
*Practically same config/setup as Client #1

  • Running as a VM (Host is running LinuxMint 20/Ubuntu 20.04)
  • OPNsense with ZT 1.6.5
  • Ethernet port used for WAN interface is dedicated to VM (passthrough)

All clients are connected to ZT and the ZT admin dashboard show them all as ONLINE. However, all clients can ping each other except Client 2. Only Client 1 can ping Client 2 and only if Client 2 pings Client 1 first.

I tried to set “allowSecondaryPort”: false on Client 2 to fix it to UDP/9993, but it did not improve the situation.

I appreciate any pointers/ideas on where to look/troubleshoot.

— mike t.

2

This seems to be a thing that others have experienced:

https://discuss.zerotier.com/t/solved-four-devices-but-two-of-them-cannot-see-each-other/1459/3
https://discuss.zerotier.com/t/inability-to-connect-to-one-machine-but-others-are-fine-ubuntu/1795
https://discuss.zerotier.com/t/cant-ping-devices/1144

There is a temporary solution in the first link. It did not work for all clients. I’ll try to “refresh” all the client in the network. Periodically doing this would seem impractical beyond a certain network size though. Has an bug/issue been filed for this?

3

Posting the solution in case someone else finds this useful.

The issue was with OPNsense all along. The default, out-of-the-box behavior is to force each interface to only 1 gateway within the interface’s subnet. So all the other gateways in the same interface were being ignored, including the firewall rules that routes traffic to the proper gateways.

You can disable this in Firewall > Settings > Advanced and tick/check/enable “Disable force gateway”. Don’t forget to save and apply this change.

1 Like
4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.