Dimensionality reduction with prior knowledge of colinearity between features
One option is to use Multidimensional scaling (MDS) for dimensionality reduction. MDS can create a visualization of the relative positions for data based on the distance between the data points.

In your example, the data points that have no distance between them in several dimensions will be projected close to each other.