Clustering big data by reducing data accuracy? I have 1 million rows with 20 attributes to do hierarchical clustering. When I want to build a distance matrix on this data by dist() in R, it says that it needs 5 TB memory. I have these approaches:


Reduce the number of rows by sampling
Change the method of clustering
?


Now, do you suggest another approach?
And I have an idea, I think if I reduce the accuracy of the values and then doing "group by", then I can remove duplicated rows and have a new column with the count of duplicates for each row. 
Is there any R package that can do hierarchical clustering with these data?

"group by": count number of duplicated rows and add a column that say how many times this row was duplicated in source.
You have n=20000000 data points and d=20 attributes.

Approach 1

The hierarchical agglomerative clustering has a time complexity of O(n^3)
and requires O(n^2) memory. Therefore it is infeasible for large data sets (not to mention big data)

What you can do is first use some nearly-linear-time clustering algorithm to cluster your data to, let as say, 2000 clusters. This algorithm can be, for example, K-Means. See this stackoverflow answer for its time complexity. This is actually an alternative to your discussion of 'reduce accuracy and group'. Then you apply hierarchical clustering to the found 2000 clusters.

Approach 2
Use the algorithm, which is specially designed for large data.
For example, Birch algorithm