What should be the value of batch_size in fit() method when using sgd (Stochastic Gradient Descent) as the optimizer? I am confused about the batch size of this model. I have used sgd i.e., Stochastic Gradient Descent as the optimizer (see the code). I am aware that in sgd, a single random instance from the training set is used to compute the gradient at each step. So, according to it, the batch_size should be equal to 1. Now, in the tf.keras.Sequential.fit() documentation it says:


  If unspecified, batch_size will default to 32.


So, do I have to manually set the batch_size equal to 1? It is because the default value, 32 will make it a Mini-batch Gradient Descent.

    import tensorflow as tf
    from tensorflow import keras

    fashion_mnist = keras.datasets.fashion_mnist
    (X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()

    X_valid, X_train = X_train_full[:5000]/255.0, X_train_full[5000:]/255.0
    y_valid, y_train = y_train_full[:5000], y_train_full[5000:]

    model = keras.models.Sequential()

    model.add(keras.layers.InputLayer(input_shape = [28, 28]))
    model.add(keras.layers.Flatten())
    model.add(keras.layers.Dense(300, activation = "relu"))
    model.add(keras.layers.Dense(100, activation = "relu"))
    model.add(keras.layers.Dense(10, activation = "softmax"))

    model.compile(loss = "sparse_categorical_crossentropy", optimizer = "sgd", metrics = ["accuracy"])

    history = model.fit(X_train, y_train, epochs = 30, validation_data = (X_valid, y_valid))
First, using the appropriate terminology you can say batch Stochastic Gradient Descent and batch Gradient descent are in the extreme ends, where Stochastic Gradient Descent is training with $batch size = 1$ and for batch gradient descent with $batch size=n$ where $n$ denotes the number of data points. 

In the appropriate terminology, what we are often using (and similar to your example as well) is called mini-batch gradient descent. Note that the term mini here does not mean it is necessarily very small like 4, 32 or 64 but instead can be anything bigger than $1$ but smaller than $n$. In practice, people use the term mini-batch gradient descent and stochastic gradient descent interchangeably. This is because in practice they behave similarly. 

I personally do not think that such practice (using SGD and minibatch SGD interchangeably) is bad, because I don't think that it differs a lot such that it requires a specific new term.