def tokenize(text):
    '\n        Function splits text into separate words and gets a word lowercased and removes whitespaces at the ends of a word. \n        The funtions also cleans irrelevant stopwords.\n        Input:\n        1. text: text message\n        Output:\n        1. Clean_tokens : list of tokenized clean words\n        '
    text = re.sub('[^a-zA-Z0-9]', ' ', text)
    tokens = word_tokenize(text)
    lemmatizer = WordNetLemmatizer()
    clean_tokens = []
    for tok in tokens:
        clean_tok = lemmatizer.lemmatize(tok, pos='v').lower().strip()
        clean_tokens.append(clean_tok)
    stopwords = nltk.corpus.stopwords.words('english')
    clean_tokens = [token for token in clean_tokens if (token not in stopwords)]
    return clean_tokens
Function splits text into separate words and gets a word lowercased and removes whitespaces at the ends of a word. 
The funtions also cleans irrelevant stopwords.
Input:
1. text: text message
Output:
1. Clean_tokens : list of tokenized clean words
def tokenize(text):
    '\n        Function splits text into separate words and gets a word lowercased and removes whitespaces at the ends of a word. \n        The funtions also cleans irrelevant stopwords.\n        Input:\n        1. text: text message\n        Output:\n        1. Clean_tokens : list of tokenized clean words\n        '
    text = re.sub('[^a-zA-Z0-9]', ' ', text)
    tokens = word_tokenize(text)
    lemmatizer = WordNetLemmatizer()
    clean_tokens = []
    for tok in tokens:
        clean_tok = lemmatizer.lemmatize(tok, pos='v').lower().strip()
        clean_tokens.append(clean_tok)
    stopwords = nltk.corpus.stopwords.words('english')
    clean_tokens = [token for token in clean_tokens if (token not in stopwords)]
    return clean_tokens