On windows, cannot start docker container running in WSL2: bind: An attempt was made to access a socket in a way forbidden by its access permissions

On my Windows 10 Pro machine, I have ZeroTier installed and joined to 2 ZeroTier networks. I also have Docker running in WSL2. The container maps two ports from the container using the default docker network: Ports 1521 in the container is mapped to port 1523 on the host and port 5500 is mapped to 5500.
I have a docker container that already existed prior to installing ZeroTier on this machine. When I disconnect from the 2 ZeroTier networks, the container starts just fine. But as soon as I am connected to the ZeroTier network on the host, I get the following error when trying to start the container:

Error response from daemon: Ports are not available: listen tcp 0.0.0.0:1523: bind: An attempt was made to access a socket in a way forbidden by its access permissions.

Sometimes, the error is for port 5500 instead of 1523.

As soon as I disconnect from ZeroTier networks, the container starts up just fine.
Using netstat -an, I confirmed that no process is listing or has connection on either of these 2 ports.