as you can seethe network is confident for some samples ( or moreor or lessbut less confident for others ( further experiments the following experiments will help convince you that the architecture choices you've made are all fairly reasonablealthough there' still room for improvementyou used two hidden layers try using one or three hidden layersand see how doing so affects validation and test accuracy try using layers with more hidden units or fewer hidden units units unitsand so on try using the mse loss function instead of binary_crossentropy try using the tanh activation (an activation that was popular in the early days of neural networksinstead of relu wrapping up here' what you should take away from this exampleyou usually need to do quite bit of preprocessing on your raw data in order to be able to feed it--as tensors--into neural network sequences of words can be encoded as binary vectorsbut there are other encoding optionstoo stacks of dense layers with relu activations can solve wide range of problems (including sentiment classification)and you'll likely use them frequently in binary classification problem (two output classes)your network should end with dense layer with one unit and sigmoid activationthe output of your network should be scalar between and encoding probability with such scalar sigmoid output on binary classification problemthe loss function you should use is binary_crossentropy the rmsprop optimizer is generally good enough choicewhatever your problem that' one less thing for you to worry about as they get better on their training dataneural networks eventually start overfitting and end up obtaining increasingly worse results on data they've never seen before be sure to always monitor performance on data that is outside of the training set licensed to