Core Updates after install with composer create-project drupal/drupal
You do in fact want to remove the drupal/core declaration (as it exists) in the replace section of composer.json, and then add the entry as you noted to the require section.  Then run composer update drupal/core --with-dependencies. 

You can find Drupal's official process for a Core upgrade with composer here: https://www.drupal.org/docs/8/update/update-core-via-composer-option-4