Query sharded collection returns all the results from the primary shard AND from the other shards
I finally tested again to follow exactly the example at https://docs.mongodb.com/v3.6/tutorial/convert-replica-set-to-replicated-shard-cluster/ . And this time it did work... It took some time (don't know how much exactly) for rs0 to delete the moved documents, but after a few minutes (or tens of minutes) they were not there anymore.

I have tried to remember what I did differently on the first time. I'm wondering if I did insert the test data after rs0 became a part of the shard directly on the rs0 replica set. And from what I understand we must not insert/update data on shard replica sets directly.