Vectorising categorical dataset for XGBoost in Sagemaker
Ok, after pulling my hair for a while, we built a solution as below:
On the preprocessing step, we used TFIDF to vectorize categorical values and then stored the data into S3.
On the training step, we read that training input and fed it into the XGB estimator.
When we train an XGB model using AWS built-in models
e.g. (container = sagemaker.image_uris.retrieve("xgboost", region, "1.2-1")),
Based on my understanding, The training job requires numerical vectors for the train and validation.
Meaning that if you have a dataset with categorical values and strings, you need to convert them into a vector. the model only deals with float numbers,
(Outside Sagemaker, I can use TFIDF to vectorize my features and construct a DMatrix), but this approach doesn't seem to be supported by Sagemaker.

Does anyone know how this data transformation is done in Sagemaker?
Is this a bad idea to use BlazyngText unsupervised learning to generate the vectors?
Should we have a preprocessing step and in that step we use TFIDF?