What is the best way to encode an arbitrary collection of strings into int categorical variables?
Let's see. You want to be able to work with unspecified number of categories.
There are multiple ways to work with this.


Create a special category "other" and put all categories which are very rare into this category during processing. When you encounter a new category, put it into this "other" category. And you can use any common preprocessing - label encoding, one hot encoding and so on. This way you'll be able to make predictions for new categories. And when you have enough data for them, you can leave then as they are and refit you preprocessing and the model.
Target encoding. There are multiple ways to convert categories to float numbers. You train target encoding on the data which you have and can apply it to new categories (usually unknown values are assigned a global mean value).