Size of Output vector from AvgW2V Vectorizer is less than Size of Input data Hi,
I have been seeing this problem for quite some time. Whenever I tried vectorizing input text data though avgw2v vectorization technique. The size of vectorized data is less than the size of the input data. Is there any statistical reason behind this? In my case 100K is the size of the input and it gives 999,98 sized output
I'm wondering what is causing this problem. Thanks in advance

Code:

listofsentences=[]
for sent in x_train:
    listofsentences.append(sent.split())

training_model = Word2Vec(sentences=listofsentences, workers=-1,min_count=5)
modelwords = list(training_model.wv.vocab)

std_avgw2v_x_train = []
for everysentence in tqdm(listofsentences):
    count = 0
    sentence = np.zeros(100)
    for everyword in everysentence:
        if everyword in modelwords:
            w2v = training_model.wv[everyword]
            count += 1
            sentence += w2v

    if count != 0:
        sentence/=count
        std_avgw2v_x_train.append(sentence)

len(std_avgw2v_x_train)
&gt;99998

len(x_train)
&gt;100000


EDIT1: I'd like to mention that I Just started learning ML, Its been 55 days since I started. Also, the same code gives our 100K output samples While I vectorize with TFIDFW2V

I have attached the image of the same. Kindly look into it
I think the issue can be one of the two :

A . You have missing value in x_train

B . One of the values in x_train has no word that is there in modelwords.

In both the cases ,

if everyword in modelwords:
     w2v = training_model.wv[everyword]
     count += 1
     sentence += w2v

condition doesn't get satisfied and you end up not addding any new value to sentence