How do I change hashing algorithm? We are transferring our old data to Drupal 8. Our users' password has been encrypted by MD5 and Drupal 8 hashing system works with SHA algorithm.

Which file should I edit to change the hashing system to the simple MD5? 

We are aware of risk, but it has been decided.
In Drupal 8.1+, password hashing and checking is managed by a service that can be replaced in your site's services.yml file.  You need to provide your own implementation of PasswordInterface.

Instead of replacing the hash method, it would be safer for your implementation to inherit from PhpassHashedPassword and override the check() method to accept your MD5 hashes (but otherwise use the parent implementation). You can use a dedicated prefix to detect old hashes. And thanks to the PhpassHashedPassword::needsRehash($hash), your password will be securely rehashed over time.

For instance, the following code expect you old hashes to be prefixed with BAD$ in the database.

class MyPasswordService extends PhpassHashedPassword {
  function check ($password, $hash) {
    if (substr($hash, 0, 4) == 'BAD$') {
      // Migrate MD5 has
      $stored_hash = substr($hash, 4);
      $computed_hash = ... old and insecure hash code ...;
      return $computed_hash &amp;&amp; Crypt::hashEquals($stored_hash, $computed_hash);
    }
    return parent::check($password, $hash);
  }
}