Hi all,
Have been using ZeroTier to access a remote site and all the clients at that site via a Teltonika RUT.
This has been incredible - I have managed routes set up so that the subnet range for the remote site goes through the Teltonika RUT, and I can interact with all the clients from my home network (or anywhere).
I noticed the “ethernet bridging” option in ZeroTier Central when looking at settings for the Teltonika RUT. Reading the description of this feature it seems like it’s there to provide what I have achieved with the managed routes I mentioned above. That is - forward my queries for clients in the remote subnet to via the RUT.
I get that managed routes allows me to specify specific clients or ranges of clients - that is useful and I do use it to allow remote support contractors to access only the client I want them to access. But is bridging the same effectively - it just provides wholesale access to the LAN behind the RUT? Or are there other subtle differences, like how broadcasts are sent through?
Can someone help explain how managed routes method is different to the ethernet bridging method?
Why would I choose one over the other? Obviously enabling ethernet bridging is slightly simpler than setting up managed routes…
I would love a comprehensive answer to this as well. The documentation shows how to do each, but says nothing about why to choose one or the other.
It does specify that bridging is L2 and managed routes is L3, so I think you’re right about broadcast differences but I’m not well-informed on that subject.
As an aside, did you not need to enable the “Allow ethernet bridging” option even when using managed routes with your Teltonika router? The guide for doing this with an OpenWrt router is very explicit about enabling that option and I’ve never even tried turning it off to see what happened.
Routing and bridging can be use to accomplish some of the same things, but they shouldn’t be used to accomplish many of those same things.
For almost all deployments, routing is how you’ll want to configure a multi-site setup. You should only configure bridging when you actually need to use a service that only operates at layer 2. Very few modern services are used that actually require strict layer 2 connectivity, and fewer still that you would use over an L2VPN like ZeroTier in bridged mode.
Stretching layer 2 across a WAN has some inherit dangers and should be done cautiously and only when necessary for specific traffic to work:
Loops
Loops in a layer 3 network generally prevent the traffic that is looping from functioning, and only that traffic. The Layer 3 Time to Live (TTL) prevents that traffic from causing drastic issues within the network as the packet is dropped once the TTL expires.
Compare that to a Layer 2 network, where no inherit packet level function exists to stop that loop. It relies on network level functions like spanning tree to stop loops…something that is likely not going to be in most people’s deployments. Unlike Layer 3, where a loop will only impact the specific looped traffic, a loop in Layer 2 will bring down the entire bridged segment and will not stop until the loop condition is fixed (e.g. infinite until the condition is remedied).
Broadcast Storms:
Something as simple as a faulty NIC on a single device can cause a broadcast storm. In a Layer 3 setup, the broadcast storm will bring down a specific segment, at a specific site. In a Bridged environment, it can bring down every segment at every site. If you have 30 sites in a bridged setup, that single faulty NIC can bring down all 30 sites.
None of this is to scare anyone away from using Bridging. The fact that ZeroTier has made it exceptionally easy to stretch Layer 2 between sites is amazing, but people should use that feature cautiously, and should not use it when the problem can be solved via routing as well.
Any insight on exactly what the “Allow ethernet bridging” setting does, and why it should or should not be required for the routing option (referring again to the ZeroTier on OpenWrt instructions which give a routed solution but explicitly say to enable that option)?
It isn’t required for the routing setting at all. Any time you’re doing a routed setup, it won’t be required.
As far as what the setting is doing under the hood, I’m not sure. If I had to guess, by default ZeroTier may only allow a single MAC address coming from a ZT interface (the MAC of the ZT interface itself). Allowing bridging could allow for additional MAC addresses to come through the interface. That’s just a guess though.