Web Crawl Dataset - Legal & Practical feedback
There seem to me to be two issues here: copyright and reproducibility of research. You pose them as being in conflict, but I don't think that is the case. 

First, in most jurisdictions, publishing someone's copyright work is a violation. "Just for research purposes" probably won't save you in these times, though once it might have done so. 

However, the reproducibility issue isn't quite as you pose it here. Suppose I publish an algorithm (I'm assuming it is an algorithm, actually - deterministic). Suppose I also publish the data set on which it is run. If someone else runs my algorithm on my data set does anyone expect that it won't produce the same result. Even a randomized "algorithm" should produce the same results within statistical limits when run on the same data set. 

But that isn't really the meaning of reproducible. I make an hypothesis about what should occur in a situation. I test it. Others should be able to test it under similar constraints. But I doubt that my hypothesis was "X happens in Y dataset with Z methodology". More likely Y here is treated as a category of datasets, not a specific one. 

So, if you publish the mechanism by which you gather the dataset, rather than the dataset itself (along with Z), then others can test your work and the results will be more meaningful than if you publish the dataset itself. Thus you avoid the copyright problem altogether. 

One reason that authors don't want to publish some datasets is to avoid stepping in quagmires such as copyright. Others involve human subject privacy, etc. But, that shouldn't matter if the hypothesis isn't overly tied to a specific dataset, but to a category of similarly gathered datasets.