Building a large distance matrix
Have you considered that the following steps will be even worse?

The standard algorithm for hierarchical clustering scales O(n³). You just don't want to use it on large data. It does not scale.

Why don't you do a simple experiment yourself: measure the time to compute the distances (and do the clustering) for n=1000,2000,4000,8000,16000,32000 and then estimate how long it will take you to process the entire data set assuming that you had enough memory... You will see that it is not feasible to use this algorithm on such big data.

You should rather reconsider your approach...