Logistic Regression - Coefficients have p-value more than alpha(0.05)
The p-value summarises a statistical test for a coefficient not to be statistically different from zero. So basically, when the p-value is > 5%, the estimated coefficient can be positive or negative (the confidence intervall includes positive and negative values). 

Often this is interpreted in a way that some variable does not make a reliable contribution to a model. This usually is relevant in causal modelling. But this does not mean that the variable does not contribute to a model and should be excluded. Omitted variable bias can be a massive problem!!! (see: Econometric Theory and Methods, Davidson/Mackinnon, Chapter 3.2 for the OLS case - very interesting stuff). Also in joint significance tests, a non-significant variable can still be relevant. 

Especially if you are interested in predictions, the p-value is not so much of a problem. In tendency you can say that overspecification of a model is less harmful than underspecification. In doubt, keep the non-significant variable in the model (it may only be a weak predictor).

However, if you have weak predictors you may also look towards Lasso or Ridge-Regression. In this regression types, variables with little or no contribution to prediction are "shrunken". This is a really cool thing to do and might be the first next thing for you to look at: https://web.stanford.edu/~hastie/glmnet/glmnet_alpha.html#log

I like Lasso/Ridge, because model selection is (at least to some extent) data driven in this case.

Also have a look at "An Introduction to Statistical Learning
with Applications in R" (Chapter 6.2). You may find a copy of the book online. The R code examples in the book are really instructive.