alternative to one hot encoding Hi I am working on a movie recommendation system and I have like 80.000 movies in my data set. The problem is there are like thousands of directors which is a problem when I represent a movie as a vector. One of the features of a movie is its director and I do not want my movie vector to be like 5000 in length. One hot encoding is advised everywhere but I want to do something else that will make my movie vectors small in length. I heard integer encoding but I have never seen an example of it nor it is advised. One thing that comes to my mind is to create a small interval like from 0 to 5 and let's say I have 6000 directors and I will assing first director to 0 and the next director will be assigned to 0+6/6000 and third one will be assigned to 0+2×6/6000 etc. However, I have a lot of doubts. what can you suggest? I am a rookie and not well-versed on the topic.
There are many other options instead of doing one-hot encoding. You can do target encoding, count encoding and many others. Here is a nice library for encoding categorical variables. Explore all the operations in here. Here's the link.
Target encoding would be my bet.