# Text preprocessing

This tokenizer has been trained with tweets that have been preprocessed as follows:

1) User mentions (@user_name) have been replaced with the word *user*.
2) URLs have been replace with the word *url*.
3) WIP.

If you are going to use this tokenizer, we recommend you to preprocess your own dataset in the same manner.
{}
region:us
TAGS
#region-us 

# Text preprocessing

This tokenizer has been trained with tweets that have been preprocessed as follows:

1) User mentions (@user_name) have been replaced with the word *user*.
2) URLs have been replace with the word *url*.
3) WIP.

If you are going to use this tokenizer, we recommend you to preprocess your own dataset in the same manner.
# Text preprocessing

This tokenizer has been trained with tweets that have been preprocessed as follows:

1) User mentions (@user_name) have been replaced with the word *user*.
2) URLs have been replace with the word *url*.
3) WIP.

If you are going to use this tokenizer, we recommend you to preprocess your own dataset in the same manner.
TAGS
#region-us 
 # Text preprocessing

This tokenizer has been trained with tweets that have been preprocessed as follows:

1) User mentions (@user_name) have been replaced with the word *user*.
2) URLs have been replace with the word *url*.
3) WIP.

If you are going to use this tokenizer, we recommend you to preprocess your own dataset in the same manner.
passage: TAGS
#region-us 
# Text preprocessing

This tokenizer has been trained with tweets that have been preprocessed as follows:

1) User mentions (@user_name) have been replaced with the word *user*.
2) URLs have been replace with the word *url*.
3) WIP.

If you are going to use this tokenizer, we recommend you to preprocess your own dataset in the same manner.