Simplest version of anapanasati
The SN 54.3 linked is not a "simple version". The link is merely abbreviated because the 16 stages are listed in previous suttas.

If per SN 47.20, mindfulness is keeping the mind on the task, since the body breathes automatically via its own nature, how can breathing be a ‘task’ to keep the mind on? There can be no such thing as mindfulness ‘of’ breathing. 

As for Sujato’s translation of “letting go”, this is the 16th step. Letting go of the 16th step is a result of 13th step experiencing impermanence. The simultaneous knowing of breathing occurs in the background.

The Pali here is paṭinissaggā. When impermanence &amp; not-self is clearly experienced at 13th step, the 16th step is paṭinissaggā, which means to give up viewing anything as self. Ajahn Buddhadasa translates it as throwing or tossing back. 

Paṭinissaggā above is the final step of practice. However, the type of letting go that serves as the initial step of practice is called vossagga. MN 118 at the end says each factor of enlightenment matures as vossagga. SN 48.9 &amp; 10 say jhana is reached by making  vossagga the meditation object.

Vossagga is letting go of craving &amp; ambition. This is the supramundane meditation object or task of mindfulness.

If you try to focus on the breathing, the progress will be very limited. The focus is on vossagga. The task is vossagga. Mindfulness is mindful of vossagga.

When there is mindfulness of vossagga, the knowing of the breathing will occur automatically, without an act of will (AN 11.2). This is why anapanasati means mindfulness with breathing.