How do I download specific blocks directly from peers, without downloading the entire chain?
You could use the pruning mode to only store blocks after the height you are interested in. I think you'll still have to download it to verify the past history, but you won't have to store it all on disk. 

See this RPC call for more information.