High model accuracy vs very low validation accuarcy I'm building a sentiment analysis program in python using Keras Sequential model for deep learning

my data is 20,000 tweets:


positive tweets: 9152 tweets
negative tweets: 10849 tweets


I wrote a sequential model script to make the binary classification as follows:

model=Sequential()
model.add(Embedding(vocab_size, 100, input_length=max_words))
model.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(250, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Fit the model

print(model.summary())
history=model.fit(X_train[train], y1[train], validation_split=0.30,epochs=2, batch_size=128,verbose=2)


however I get very strange results! The model accuracy is almost perfect (>90)
whereas the validation accuracy is very low (&lt;1) (shown bellow)

Train on 9417 samples, validate on 4036 samples
 Epoch 1/2
- 13s - loss: 0.5478 - acc: 0.7133 - val_loss: 3.6157 - val_acc: 0.0243
 Epoch 2/2
- 11s - loss: 0.2287 - acc: 0.8995 - val_loss: 5.4746 - val_acc: 0.0339


I tried to increase the number of epoch, and it only increases the model accuracy and lowers the validation accuracy

Any advice on how to overcome this issue?

Update:

this is how I handle my data

#read training data
pos_file=open('pos2.txt', 'r', encoding="Latin-1")
 neg_file=open('neg3.txt', 'r', encoding="Latin-1")
# Load data from files
pos = list(pos_file.readlines())
neg = list(neg_file.readlines()) 
x = pos + neg
docs = numpy.array(x)
#read Testing Data
pos_test=open('posTest2.txt', 'r',encoding="Latin-1")
posT = list(pos_test.readlines())
neg_test=open('negTest2.txt', 'r',encoding="Latin-1")
negT = list(neg_test.readlines())
xTest = posT + negT
total2 = numpy.array(xTest)

CombinedDocs=numpy.append(total2,docs)

# Generate labels
positive_labels = [1 for _ in pos]
negative_labels = [0 for _ in neg]
labels = numpy.concatenate([positive_labels, negative_labels], 0)

# prepare tokenizer
t = Tokenizer()
t.fit_on_texts(CombinedDocs)
vocab_size = len(t.word_index) + 1
# integer encode the documents
encoded_docs = t.texts_to_sequences(docs)
#print(encoded_docs)

# pad documents to a max length of 140 words
max_length = 140
padded_docs = pad_sequences(encoded_docs, maxlen=max_length, padding='post')


Here I used Google public word2vec

# load the whole embedding into memory
embeddings_index = dict()
f = open('Google28.bin',encoding="latin-1")
for line in f:
values = line.split()
word = values[0]
coefs = asarray(values[1:], dtype='str')
embeddings_index[word] = coefs
f.close()

# create a weight matrix for words in training docs
embedding_matrix = zeros((vocab_size, 100))

for word, i in t.word_index.items():
embedding_vector = embeddings_index.get(word)
if embedding_vector is not None:
    embedding_matrix[i] = embedding_vector


#Convert to numpy
NewTraining=numpy.array(padded_docs)
NewLabels=numpy.array(labels)
encoded_docs2 = t.texts_to_sequences(total2)

# pad documents to a max length of 140 words

padded_docs2 = pad_sequences(encoded_docs2, maxlen=max_length, padding='post')


# Generate labels
positive_labels2 = [1 for _ in posT]
negative_labels2 = [0 for _ in negT]
yTest = numpy.concatenate([positive_labels2, negative_labels2], 0)
NewTesting=numpy.array(padded_docs2)
NewLabelsTsting=numpy.array(yTest)
When a machine learning model has high training accuracy and very low validation then this case is probably known as over-fitting. The reasons for this can be as follows:


The hypothesis function you are using is too complex that your model perfectly fits the training data but fails to do on test/validation data.
The number of learning parameters in your model is way too big that instead of generalizing the examples , your model learns those examples and hence the model performs badly on test/validation data.


To solve the above problems a number of solutions can be tried depending on your dataset:


Use a simple cost and loss function.
Use regulation which helps in reducing over-fitting i.e Dropout.
Reduce the number of learning parameters in your model.


These are the 3 solutions that are most likely to improve the validation accuracy of your model and still if these don't work check your inputs whether they have the right shapes and sizes.