Local minima in Stochastic Gradient Descent I am running a logisitic regression model using Keras on a dataset. However, I am often running in a problem in which the model settle at an validation accuracy value lower than a theoretical value. 

The hypothesis is confirmed by the fact that if I run the model again (without changing anything), the model settles at a different value (often higher) for validation. 

I presume such a problem is due to the gradient descent encountering a local minimum and when I run it again, due to random fluctuations, I sometimes end up skipping the local minima. 

So any method, to prevent such a problem?

def base():
    model = Sequential()
    model.add(Dense(20,activation = 'linear', input_dim = 12288))
    model.add(Dense(7,activation = 'relu'))
    model.add(Dense(5, activation = 'linear'))
    model.add(Dense(1,activation = 'sigmoid'))
    model.compile(optimizer = optimizers.SGD(lr=0.0075), loss = 'binary_crossentropy', metrics= ['accuracy'])      
    return model 

model = base()
model.fit(train_x,train_y, epochs = 2400, batch_size =10,validation_data = [test_x, test_y], verbose = 2)
One of the major advantage of stochastic gradient descent (SGD) over batch descent is the ability to explore different local minima's. But due to stochastic nature, you might end up with different results every time you repeat. 

In all regression the most important task is to perform regularization to achieve a generalized fit for better cross validation scores. Your issue might be of over fitting rather than stochastic nature of optimizer. In your model.add method, add an argument for kernel_regularizer=L1L2(l1=0.0, l2=0.15). Try with different values for shrinkage of L1, L2 norms.

Also, If you are looking for stability in accuracy you can try batch/ vanilla gradient descent. Also, I would recommend you to try different learning rates and other optimizer like Adam that make use of adaptive learning rates.