What algorithm should I use to find which sub-component is associated with a attribute? I have a n x m one hot encoded matrix of a number of units, where n corresponds to the number of units, and m is the number of total sub-components. This corresponds to a n x 5  matrix which has the 5 attribute values for each of these units.

If I want to create clusters of these sub-components corresponding to these 5 attributes (basically see which sub-component is associated with which attribute), which technique should I use? Would something like PCA or t-SNE work or does this problem require a different approach and why should I use what I use?

I am quite new to this, I would appreciate any help on this matter.
The type of clustering depends on the measures scale of the data. You stated you have 5 attribute values. Are those values nominal, ordinal, interval, or ratio?
If they are nominal, correspondence analysis is a common option.
If they are ordinal, k-medoids clustering could work.
If they are interval or ratio, k-means clustering is a very popular choice.