I was thinking a bit about ZT and was wondering whether, hypothetically, encryption overhead between two ZT nodes and between a given host and a speedtest server (e.g. fast.com) using https (say with TLS) should be roughly equivalent, all things equal (e.g. say in this imaginary world fast.com is also hosted locally in the same LAN as the ZT nodes).
To my understanding, both use asymmetric key encryption for initial handshaking, then symmetric key encryption for the actual conversation. So from a systems design perspective, there’s “one” encryption and decryption operation between application space (in the ZT case zerotier-one, and in the fast dotcom case the browser/OpenSSL) and the wire, assuming you’re not doing https over ZT - is that right?
The reason I ask is that on a Linux host I’m noticing about 780Mbps on a speed test to fast dotcom via fiber WAN, while running an iperf3 test to another Linux host on the same Gigabit LAN (connected via switch) through ZT (running on both hosts) I’m able to achieve about 300Mbps. At first I thought the difference in speed was primarily encryption overhead, but thinking about it some more second-guessed and came up with the above.
Encryption should be done in hardware on anything with AES acceleration.
io is single threaded at the moment. It can only go as fast as one of your cores. Improvements coming soon ™️
Is zerotier using 100% of a cpu during iperf3 tests?
Hi there, thanks for the speedy response. Yeah, ZT is maxing out one of the CPU cores during the iperf3 tests (as is iperf3, which to my understanding is also single-threaded). The hardware I’m using has AES-acceleration capability (ARM-based SBC with AES extensions, confirmed by grep’ing /proc/cpuinfo) so I’m guessing that’s being utilized per some of the code I’ve browsed on the ZT GitHub repo that shows AES instructions being used if the CPU has AES-accelerated hardware capabilities.
Good to know re: ZT goes as fast as one of your cores. Is the multithreading coming with ZT 2.0, or might that make it into the 1.x series prior?
@Boilerplate4U for sure - with a trusted path setup between the two Linux nodes, there’s an ~10% improvement in the throughput of the iperf3 test. So just to summarize:
Great thanks, the aes offload encryption seem to perform really well thus throughput is cpu bound just like zt-travis explained. What hardware was this tested on?