Fast Upload, Slow Download on Wireless-AC WAN over SFTP
Maybe the site/computer you are downloading from have a upload limit. but when you push files (you upload) the other side is downloading, so it less of a problem for them.