How do I manually update without using Drush nor Composer?
The official update method is described in UPDATE.txt:

http://cgit.drupalcode.org/drupal/tree/core/UPDATE.txt?h=8.4.x

This is basically what you've described in the question. You don't need composer to update core.

If you want to switch your existing installation to composer a risk free way would be to start a new project, see Updating core with Composer doesn&#39;t work.