SRX NAT configuration for a ZT appliance

[jkhuon]

I am deploying multiple branch office sites where I intend to use ZeroTier to host an interoffice VPN. My topology for each site is as follows:

• Dual/multiple providers connected into a VR routing-instance called VR-Internet on an SRX240. Each provider participates in eBGP and I announce my PI public IP space to them.
• A ZeroTier client is hosted on a small server with multiple ports connected to the SRX240. One port lands in a VR I call EdgeNAT. The ZT appliance’s interface in that routing-instance is configured with a private address. This VR has NAT enabled between it and VR-Internet and routing is enabled between the two routing-instances.
• A second interface on the ZT appliance lands in another routing-instance called VR-VPN. This interface is configured as a bridge. Thus the ZT LAN IP layer3 is configured on the SRX240 itself.
• VR-VPN is stretched down to a Juniper EX4200 switch where it lands in a routing-instance configured as a VRF called VRF-VPN.
• The switch also hosts several other VRF constructs such as VRF-Office for the internal network of the branch office and VRF-Mgmt for network/server/facilities management. The switch essentially serves as sitewide intraoffice core switch and punts inter-office as well as Internet traffic to the SRX240 firewall.

I am observing the following:

Everything works fine in the beginning. A host in VRF-Office in Office-A is able to reach another host in VRF-Office in Office-B. However, the VRFs become unreachable after about 2 hours. During this time I can ping all ZT-LAN endpoints fine. Restarting the zerotier-one client or rebooting the server restores connectivity for the VRFs sitting behind the edge.

Other things to note:

I have permit rules for port 9993/udp and ephemeral UDP ports >=1024. Timeouts were originally set to 60s. I tried changing the timeouts to “never” but that didn’t seem to have any effect.

I suspect I am not configuring NAT on the SRX240 properly but I’m unsure what the configuration should be.The following is my current pertinent NAT configuration:

security { ...
... }
    nat {
        source {
            rule-set EdgeNAT {
                from zone EdgeNAT;
                to zone Internet;
                rule EdgeNAT_to_Internet {
                    match {
                        source-address 192.168.5.0/24;
                    }
                    then {
                        source-nat {
                            interface;
                        }
                    }
                }
            }
        }
... }

Any advice would be appreciated.

[zt-travis]

Hello
Very interesting. Thanks for writing.
What OS is the VM zerotier is running on? Sorry not super familiar with Juniper stuff.

Does Juniper NAT randomize ports? If so, can it be configured not to do that?

The 2 hours part is kind of strange. When the zerotier service is restarted, it binds to different random ports. That it starts working again after a restart may point to something.

zerotier-cli peers or zerotier-cli peers -j will show the paths to the other peers. You may be able to compare this with the current state of the NAT, if there’s a way to print that.

Could potentially be some kind of IDS getting triggered.

Those are the first guesses that come to mind.
Let us know!

[jkhuon]

The ZT appliance are running ubuntu 20.04 LTS. Yeah… I believe the Juniper was randomizing ports given the config from my original post. I think I just managed to fix this with explicit persistence.

source {
    pool ZT {
        address {
            1.2.3.4/32;
        }
        port {
            no-translation;
        }
    }
    { ... }
    rule-set EdgeNAT {
        from zone EdgeNAT;
        to zone Internet;
        rule EdgeNAT_to_Internet {
            match {
                source-address 192.168.5.10/32;
            }
            then {
                source-nat {
                    pool {
                        ZT;
                        persistent-nat {
                            permit any-remote-host;
                            address-mapping;
                            max-session-number 65536;
                        }
                    }
                }
            }
        }
        rule default {
            match {
                source-address 192.168.5.0/24;
            }
            then {
                source-nat {
                    interface;
                }
            }
        }
    }
}

I’ve pushed past the 2-hour mark and things still seem stable.

[zt-travis]

awesome. Thanks for posting the config!