What are the best activation functions for Binary text classification in neural networks?
Relu often works well, but ultimately you need to try. If you really have only one hidden layer, you might have a model which cannot properly learn. 

I guess you use a sequential, dense model architecture? Try to add more hidden layers. Don‘t add too much capacity for a start. You could try something like 256, 8, 8, 1 with relu. 

Dropout is good to prevent overfitting. It is not important for learning per se.

Just a remark for future posts: always include your basic model code if possible. So people can understand what you are really up to. Also a little data description is useful. Especially with text, preprocessing is really important. So the way you treated your data before it goes into the model is relevant.