How does the brain instinctively know the math behind Newtonian Physics?
Trial and error.

The CNS is very much a living, changing organ system - much more so before you are an adult. While you are a child your neurons fire, muscles contract, and force is exerted on objects - then your CNS modifies itself (either through redistribution of neural contacts or growth of new neurons spurned by chemical changes) to account for the results.

This is why "practice makes perfect." As you continuously perform a task, like throwing a ball, your body is keeping track of the results. Did X muscle fibers contracting in Y muscles overshoot your buddy? You register the failure and compensate by utilizing X-N muscle fibers that contract in Y muscles until you consistently satisfy your goal.

Your CNS is keeping track of everything, all the time. There isn't a single instant in your life - asleep or awake - that's not being monitored, so your body has copious amounts of data to work with when making its estimates.

This is also why simulations take time to adapt to, even if the person is fully aware of the basics behind the physics involved.