Feature selection or Dimension reduction in unsupervised learning
I found Auto-encoders to be the best solution for this.
Performing auto encoder before clustering reduces the dimensionality of the high dimensional data and then the encoder results can be extracted and used for whatever method we want to implement like,

supervised classification
unsupervised clustering etc.

Autoencoders using CNN (encoder and decoder) for image:
https://www.datacamp.com/community/tutorials/autoencoder-keras-tutorial#comment-6125

Autoencoders - its types and usage - LSTM:
https://machinelearningmastery.com/lstm-autoencoders/

for my usecase I should be performing an embedding before feeding it into the autoencoder (CNN/LSTM) with only the encoder so that the reduced dimensions can be used for kmeans clustering.

Hope this helps people who have same question, I wonder how none has come across this situation or haven't replied for this.

Thanks
Arav