Understanding how the loss was calculated for the SQuAD task in BERT paper
These answers are based on my personal understanding of Bert from both the paper and official_implementation, hope it will help:

What do they mean by "maximum scoring span is used as the prediction"?


As you know in SQuAD the input sequence is divided to 2 parts: Question and Document (from which we extract the answer if possible). 

Sometimes the input length exceeds the max_seq_length parameter, in this case the document is truncated to as many parts as needed and we end up having more than one input for the same question/document. Mention that the question is replicated in all the resulting inputs (see line_350 for details). 

So in such cases, in order to determine the predicted span among all the generated-inputs we use the maximum scoring i.e (max_start + max_end) / 2 = ( max(Softmax(S*Ti)) + max(Softmax(E*Ti)) ) / 2 of all the inputs related to the same question.

"The training objective is the loglikelihood of the correct start and end positions"


The loss is the average of the start_position loss start_loss and the end_postition loss end_loss. Each loss is computed in the same way: after applying the Softmax to the final output (i.e S*Ti or E*Ti) we use the real start/end postitions to compute the loss (see code below).

From run_squad.py in Bert_repo:

 def compute_loss(logits, positions):
        one_hot_positions = tf.one_hot(positions, depth=seq_length, dtype=tf.float32)
        log_probs = tf.nn.log_softmax(logits, axis=-1)
        loss = -tf.reduce_mean(tf.reduce_sum(one_hot_positions * log_probs, axis=-1))
        return loss

 start_positions = features["start_positions"]
 end_positions = features["end_positions"]

 start_loss = compute_loss(start_logits, start_positions)
 end_loss = compute_loss(end_logits, end_positions)

 total_loss = (start_loss + end_loss) / 2.0