Bridge not working Ubuntu 20 <-> OPNSense

[lfstudios10]

I have an Ubuntu 20 VM setup and I followed these directions: https://zerotier.atlassian.net/wiki/spaces/SD/pages/7471125/Layer+2+Bridging+of+Ethernet+and+ZeroTier+Networks+on+Linux

The Ubuntu 20 VM has a bridged network interface with my ‘home’ network. For experimental purposes I’m trying to see if I can get all traffic to flow over Zerotier

On the other end I have a Protectcli box running OPNSense with one of the ports bridged to the LAN. I’d imagine, if I had done this correctly, I should be receiving a DHCP address from my ‘home’ router however no IP address is to be had. I connected a computer to the same Zerotier instance to see what traffic I could see and there was barely anything - I’d imagine I should be seeing a shit ton.

Both units have the allow riding checked on Zerotier.

I’m sure I’m missing something basic so hopefully someone can direct me in the right direction.

[lfstudios10]

Looks like there was an incorrect setting for the name of an adapter to bring up in the bridge. Now that the adapter is up I am seeing more traffic - still no DHCP!

[lfstudios10]

I think this is potentially related to other multicast posts @zt-travis has replied on. Hopefully they can help.

[zt-travis]

Hello
yeah… bridging can be pretty tricky to set up.

One thing to try is zerotier-cli set $networkid allowManaged=0 on the bridge nodes. The tells zerotier to stop trying to configure the interface -which is inside the bridge- and routes for that network. You can set an IP address manually on the bridge interface if you need.

I’m not sure if the default network rules would block dhcp? You can delete all the rules and just have accept;. Actually, it looks like you’ve done that.

[lfstudios10]

I appreciate you weighing-in on this.

As you mentioned, you can see that I’ve taken out all the drop commands so everything should be forwarded. I can’t imagine that the networked allow managed setting would help however I will try it. Assuming that doesn’t, do you have any other tips or diagnostics to get to the bottom of this?

[lfstudios10]

@zt-travis

networkid allowManaged=0

Following up here. Do I simply paste the command into my OpnSense router and change the network ID (see attached image). Also, on my Linux system I do not see local.conf. Am I in the correct directory? Do I simply need to create it? (also see attached picture)

[zt-travis]

Hello
I’m not familiar with the opnsense interface. What is in the networks tab of that web ui?

sudo zerotier-cli set $networkid allowManaged=0 is run from command line. ( you need to replace $networkid with your actual network id). It’s similar to zerotier-cli join $networkid
It doesn’t go in local.conf. You should leave local.conf blank for now.

[lfstudios10]

Here is the output from my Linux setup:

sudo zerotier-cli set a0cbf4b62ab51473 allowManaged=0
[sudo] password for ianliuzzi-fedun: 
{
 "allowDNS": false,
 "allowDefault": false,
 "allowGlobal": false,
 "allowManaged": false,
 "assignedAddresses": [
  "192.168.26.130/24"
 ],
 "bridge": true,
 "broadcastEnabled": true,
 "dhcp": false,
 "dns": {
  "domain": "",
  "servers": []
 },
 "id": "a0cbf4b62ab51473",
 "mac": "72:c9:b2:78:fb:c2",
 "mtu": 2800,
 "multicastSubscriptions": [
  {
   "adi": 0,
   "mac": "01:00:5e:00:00:01"
  },
  {
   "adi": 0,
   "mac": "01:00:5e:00:00:fb"
  },
  {
   "adi": 3232242306,
   "mac": "ff:ff:ff:ff:ff:ff"
  }
 ],
 "name": "Protectli",
 "netconfRevision": 18,
 "nwid": "a0cbf4b62ab51473",
 "portDeviceName": "ztc3qudisz",
 "portError": 0,
 "routes": [
  {
   "flags": 0,
   "metric": 0,
   "target": "192.168.1.0/24",
   "via": null
  },
  {
   "flags": 0,
   "metric": 0,
   "target": "192.168.25.0/24",
   "via": "192.168.26.200"
  },
  {
   "flags": 0,
   "metric": 0,
   "target": "192.168.26.0/24",
   "via": null
  }
 ],
 "status": "OK",
 "type": "PRIVATE"

I take it that since it says managed:false we’re good to go. I do need to assign an IP address within my ZT network scope, correct?

I’ve attached a picture of what is presented on the networks page OpnSene and I do not thing that is what you’re looking for.

[zt-travis]

@lfstudios10

I take it that since it says managed:false we’re good to go.

Looks good

@lfstudios10

I do need to assign an IP address within my ZT network scope, correct?

Not strictly necessary to pass bridged traffic as far as I know. You can assign an IP to the bridge interface if you need to connect to this device over zerotier.

Thanks for the screen shot. I wonder if the pencil icon has the “allow_” options.
Hope that helps!

[lfstudios10]

I figured you’d ask about the pencil. I’ve attached below but it is virtually useless.

Adding the network in the local.conf screen doesn’t really do what I need as the opnsense unit does not connect to zero tier after I do so. When running the Linux commands on the Linux box, where does it actually place that “allowManaged=0” variable? Perhaps I can do the same on my opnsense box?

As far as IPs, I need the traffic to flow between two devices as if they are on a L2 segment however the devices themselves don’t need to be part of it.

[zt-travis]

@lfstudios10

When running the Linux commands on the Linux box, where does it actually place that “allowManaged=0” variable?

It immediately changes the value in the running zerotier service. Zerotier will remove any routes and addresses for that network.

This is saved in /var/lib/zerotier-one/networks.d/$id.local.conf on linux. On bsd it’s /var/db/zerotier-one/

You can edit those files and restart zerotier-one, but I’d hope zerotier-cli works on opnsense too.

[lfstudios10]

I found that and edited it manually in OpnSense to be safe. Can you clarify if I do need to manually assign an IP address to any of the interfaces/bridges created or if that is not necessary?