Node disconnected when too many threads are tunneling

Hi, I have added 2 machines (A & B) in ZT. I’ve installed Haproxy and A to relay all the traffic towards B using managed routes. I have installed a proxy on B i.e. with the help of Haproxy, when I connect to server A, I will be redirected to server B.

The case is if I run speedtest(multi-thread mode) on my mobile devices to see how much speed I can get, server B will lose connection. The ping result is presented below. During the speedtest, destination host unreachable pops up and ofc speedtest is stopped.

Server A:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 78:45:c4:fa:87:05 brd ff:ff:ff:ff:ff:ff
    inet real IP here/24 brd 120.241.33.255 scope global em1
       valid_lft forever preferred_lft forever
    inet6 fe80::7a45:c4ff:fefa:8705/64 scope link 
       valid_lft forever preferred_lft forever
3: em2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:45:c4:fa:87:06 brd ff:ff:ff:ff:ff:ff
11: ztppi5o: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2800 qdisc pfifo_fast state UNKNOWN qlen 500
    link/ether ba:fe:4a:59:1c:53 brd ff:ff:ff:ff:ff:ff
    inet 192.168.192.178/24 brd 192.168.192.255 scope global ztppi5osng
       valid_lft forever preferred_lft forever
    inet6 fe80::b8fe:4aff:fe59:1c53/64 scope link 
       valid_lft forever preferred_lft forever

Server B:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether d4:ae:52:c5:eb:e5 brd ff:ff:ff:ff:ff:ff
    inet realIPhere/26 brd 209.58.185.255 scope global em1
       valid_lft forever preferred_lft forever
    inet6 fe80::d6ae:52ff:fec5:ebe5/64 scope link 
       valid_lft forever preferred_lft forever
3: em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether d4:ae:52:c5:eb:e6 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::d6ae:52ff:fec5:ebe6/64 scope link 
       valid_lft forever preferred_lft forever
5: ztppi5o: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2800 qdisc pfifo_fast state UNKNOWN group default qlen 1000
    link/ether ba:5a:14:ce:8f:cf brd ff:ff:ff:ff:ff:ff
    inet 192.168.192.36/24 brd 192.168.192.255 scope global ztppi5osng
       valid_lft forever preferred_lft forever
    inet6 fe80::b85a:14ff:fece:8fcf/64 scope link 
       valid_lft forever preferred_lft forever
64 bytes from 192.168.192.36: icmp_seq=12 ttl=64 time=76.3 ms
64 bytes from 192.168.192.36: icmp_seq=13 ttl=64 time=69.9 ms
64 bytes from 192.168.192.36: icmp_seq=14 ttl=64 time=63.4 ms
64 bytes from 192.168.192.36: icmp_seq=15 ttl=64 time=15.0 ms
64 bytes from 192.168.192.36: icmp_seq=16 ttl=64 time=14.9 ms
64 bytes from 192.168.192.36: icmp_seq=17 ttl=64 time=15.0 ms
From 192.168.192.178 icmp_seq=42 Destination Host Unreachable
From 192.168.192.178 icmp_seq=45 Destination Host Unreachable
From 192.168.192.178 icmp_seq=46 Destination Host Unreachable
From 192.168.192.178 icmp_seq=47 Destination Host Unreachable
From 192.168.192.178 icmp_seq=48 Destination Host Unreachable
From 192.168.192.178 icmp_seq=49 Destination Host Unreachable

I tried different servers but all stay the same.

Tried

  1. Trusted path added
  2. New ZT networks
  3. added net.ipv4.conf.em1.rp_filter=2
  4. systemctl stop firewalld
  5. Increase Multicast Recipient Limit
  6. Disable Broadcast (ff:ff:ff:ff:ff:ff)

Question:
Any hint on this issue? Is it possible that the network provider bans this? But I tried machines from different data centres and all the same. Some data centres machines are fine, but some are not.

Welcome!

Check the CPU usage of the zerotier-one process during the test.
What is the upload speed of the phone?

Well, all the machines are dedicated servers so I guess it won’t b the cpu usage.

I have tried configured both sides of the machines trusted path and seems like it is working properly now. I will run more tests xD.

Download 250Mbps/ Upload 50Mbps