Training a neural network to output the conditional probability of an event when the training data output is only binary
To sort the terms a bit, back propagation (repeatedly applying the chain rule of differentiation) is an implementation of stochastic gradient descent, which is an implementation of optimization. It is not necessarily the square of the error that is reduced by optimization - it is up to us to choose the loss function.

The loss function should ideally mirror the utility function (what you are truly want to optimize), but it needs a useful derivative. For example, the 0-1 loss hasn't one.

I think that what is called linear confidence guess in the question is the conditional probability of the event, given the input. Then the binary cross entropy would be a valid choice of a loss function.

From deeplearningbook.org


  The negative log-likelihood allows the model to estimate the conditional probability of the classes, given the input, and if the model can do that well, then it can pick the classes that yield the least classiﬁcation error in expectation.