Logistic Regression Maximum Likelihood
A Bernoulli distribution is a probability distribution where a random variable takes the value 1 with probability p and the value 0 with probability 1-p. This distribution maps very well onto logistic regression because the random variable is the target variable and the model is estimating the probability of 0 or 1 given the specific values of the features.
The Gaussian distribution is not as well suited because it is a continuous probability distribution that is goes from negative infinity to positive infinity. The result would be predictions outside the bounds of the problem space.
The cost function is something different. The cost function is how errors in prediction are penalized.