prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  BatchNormalization does not switch the current GPU automatically
  As reported in the forum, the current implementation of BatchNormalization link does not switch the current GPU appropriately during computing the running statistics of batches.
  It causes confusing error message like `ValueError: Array device must be same as the current device: array device = 1 while current = 0`.
  See for example: https://groups.google.com/forum/#!topic/chainer/T-6s3KD-X-U
  
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `chainer/links/normalization/batch_normalization.py`
  Content:
  ```
  1 import numpy
  2 
  3 from chainer.functions.normalization import batch_normalization
  4 from chainer import link
  5 from chainer import variable
  6 
  7 
  8 class BatchNormalization(link.Link):
  9 
  10     """Batch normalization layer on outputs of linear or convolution functions.
  11 
  12     This link wraps the :func:`~chainer.functions.batch_normalization` and
  13     :func:`~chainer.functions.fixed_batch_normalization` functions.
  14 
  15     It runs in three modes: training mode, fine-tuning mode, and testing mode.
  16 
  17     In training mode, it normalizes the input by *batch statistics*. It also
  18     maintains approximated population statistics by moving averages, which can
  19     be used for instant evaluation in testing mode.
  20 
  21     In fine-tuning mode, it accumulates the input to compute *population
  22     statistics*. In order to correctly compute the population statistics, a
  23     user must use this mode to feed mini batches running through whole training
  24     dataset.
  25 
  26     In testing mode, it uses pre-computed population statistics to normalize
  27     the input variable. The population statistics is approximated if it is
  28     computed by training mode, or accurate if it is correctly computed by
  29     fine-tuning mode.
  30 
  31     Args:
  32         size (int or tuple of ints): Size (or shape) of channel
  33             dimensions.
  34         decay (float): Decay rate of moving average. It is used on training.
  35         eps (float): Epsilon value for numerical stability.
  36         dtype (numpy.dtype): Type to use in computing.
  37 
  38     See: `Batch Normalization: Accelerating Deep Network Training by Reducing\
  39           Internal Covariate Shift <http://arxiv.org/abs/1502.03167>`_
  40 
  41     .. seealso::
  42        :func:`~chainer.functions.batch_normalization`,
  43        :func:`~chainer.functions.fixed_batch_normalization`
  44 
  45     Attributes:
  46         gamma (~chainer.Variable): Scaling parameter.
  47         beta (~chainer.Variable): Shifting parameter.
  48         avg_mean (~chainer.Variable): Population mean.
  49         avg_var (~chainer.Variable): Population variance.
  50         N (int): Count of batches given for fine-tuning.
  51         decay (float): Decay rate of moving average. It is used on training.
  52         eps (float): Epsilon value for numerical stability. This value is added
  53             to the batch variances.
  54 
  55     """
  56     def __init__(self, size, decay=0.9, eps=1e-5, dtype=numpy.float32):
  57         super(BatchNormalization, self).__init__()
  58         self.add_param('gamma', size, dtype=dtype)
  59         self.gamma.data.fill(1)
  60         self.add_param('beta', size, dtype=dtype)
  61         self.beta.data.fill(0)
  62         self.add_persistent('avg_mean', numpy.zeros(size, dtype=dtype))
  63         self.add_persistent('avg_var', numpy.zeros(size, dtype=dtype))
  64         self.add_persistent('N', 0)
  65         self.decay = decay
  66         self.eps = eps
  67 
  68     def __call__(self, x, test=False, finetune=False):
  69         """Invokes the forward propagation of BatchNormalization.
  70 
  71         BatchNormalization accepts additional arguments, which controls three
  72         different running mode.
  73 
  74         Args:
  75             x (Variable): An input variable.
  76             test (bool): If ``True``, BatchNormalization runs in testing mode;
  77                 it normalizes the input using pre-computed statistics.
  78             finetune (bool): If ``True``, BatchNormalization runs in
  79                 fine-tuning mode; it accumulates the input array to compute
  80                 population statistics for normalization, and normalizes the
  81                 input using batch statistics.
  82 
  83         If ``test`` and ``finetune`` are both ``False``, then
  84         BatchNormalization runs in training mode; it computes moving averages
  85         of mean and variance for evaluation during training, and normalizes the
  86         input using batch statistics.
  87 
  88         """
  89         use_batch_mean = not test or finetune
  90 
  91         if use_batch_mean:
  92             func = batch_normalization.BatchNormalizationFunction(self.eps)
  93             ret = func(x, self.gamma, self.beta)
  94 
  95             if finetune:
  96                 self.N += 1
  97                 decay = 1. - 1. / self.N
  98             else:
  99                 decay = self.decay
  100 
  101             m = x.data.size // self.gamma.data.size
  102             adjust = m / max(m - 1., 1.)  # unbiased estimation
  103             self.avg_mean *= decay
  104             func.mean *= 1 - decay  # reuse buffer as a temporary
  105             self.avg_mean += func.mean
  106             del func.mean
  107             self.avg_var *= decay
  108             func.var *= (1 - decay) * adjust  # reuse buffer as a temporary
  109             self.avg_var += func.var
  110             del func.var
  111         else:
  112             mean = variable.Variable(self.avg_mean, volatile='auto')
  113             var = variable.Variable(self.avg_var, volatile='auto')
  114             ret = batch_normalization.fixed_batch_normalization(
  115                 x, self.gamma, self.beta, mean, var, self.eps)
  116         return ret
  117 
  118     def start_finetuning(self):
  119         """Resets the population count for collecting population statistics.
  120 
  121         This method can be skipped if it is the first time to use the
  122         fine-tuning mode. Otherwise, this method should be called before
  123         starting the fine-tuning mode again.
  124 
  125         """
  126         self.N = 0
  127
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/chainer/links/normalization/batch_normalization.py b/chainer/links/normalization/batch_normalization.py
  --- a/chainer/links/normalization/batch_normalization.py
  +++ b/chainer/links/normalization/batch_normalization.py
  @@ -1,5 +1,6 @@
   import numpy
   
  +from chainer import cuda
   from chainer.functions.normalization import batch_normalization
   from chainer import link
   from chainer import variable
  @@ -98,16 +99,17 @@
               else:
                   decay = self.decay
   
  -            m = x.data.size // self.gamma.data.size
  -            adjust = m / max(m - 1., 1.)  # unbiased estimation
  -            self.avg_mean *= decay
  -            func.mean *= 1 - decay  # reuse buffer as a temporary
  -            self.avg_mean += func.mean
  -            del func.mean
  -            self.avg_var *= decay
  -            func.var *= (1 - decay) * adjust  # reuse buffer as a temporary
  -            self.avg_var += func.var
  -            del func.var
  +            with cuda.get_device(x.data):
  +                m = x.data.size // self.gamma.data.size
  +                adjust = m / max(m - 1., 1.)  # unbiased estimation
  +                self.avg_mean *= decay
  +                func.mean *= 1 - decay  # reuse buffer as a temporary
  +                self.avg_mean += func.mean
  +                del func.mean
  +                self.avg_var *= decay
  +                func.var *= (1 - decay) * adjust  # reuse buffer as a temporary
  +                self.avg_var += func.var
  +                del func.var
           else:
               mean = variable.Variable(self.avg_mean, volatile='auto')
               var = variable.Variable(self.avg_var, volatile='auto')
  
verification_info: {"golden_diff": "diff --git a/chainer/links/normalization/batch_normalization.py b/chainer/links/normalization/batch_normalization.py\n--- a/chainer/links/normalization/batch_normalization.py\n+++ b/chainer/links/normalization/batch_normalization.py\n@@ -1,5 +1,6 @@\n import numpy\n \n+from chainer import cuda\n from chainer.functions.normalization import batch_normalization\n from chainer import link\n from chainer import variable\n@@ -98,16 +99,17 @@\n             else:\n                 decay = self.decay\n \n-            m = x.data.size // self.gamma.data.size\n-            adjust = m / max(m - 1., 1.)  # unbiased estimation\n-            self.avg_mean *= decay\n-            func.mean *= 1 - decay  # reuse buffer as a temporary\n-            self.avg_mean += func.mean\n-            del func.mean\n-            self.avg_var *= decay\n-            func.var *= (1 - decay) * adjust  # reuse buffer as a temporary\n-            self.avg_var += func.var\n-            del func.var\n+            with cuda.get_device(x.data):\n+                m = x.data.size // self.gamma.data.size\n+                adjust = m / max(m - 1., 1.)  # unbiased estimation\n+                self.avg_mean *= decay\n+                func.mean *= 1 - decay  # reuse buffer as a temporary\n+                self.avg_mean += func.mean\n+                del func.mean\n+                self.avg_var *= decay\n+                func.var *= (1 - decay) * adjust  # reuse buffer as a temporary\n+                self.avg_var += func.var\n+                del func.var\n         else:\n             mean = variable.Variable(self.avg_mean, volatile='auto')\n             var = variable.Variable(self.avg_var, volatile='auto')\n", "issue": "BatchNormalization does not switch the current GPU automatically\nAs reported in the forum, the current implementation of BatchNormalization link does not switch the current GPU appropriately during computing the running statistics of batches.\nIt causes confusing error message like `ValueError: Array device must be same as the current device: array device = 1 while current = 0`.\nSee for example: https://groups.google.com/forum/#!topic/chainer/T-6s3KD-X-U\n\n", "before_files": [{"content": "import numpy\n\nfrom chainer.functions.normalization import batch_normalization\nfrom chainer import link\nfrom chainer import variable\n\n\nclass BatchNormalization(link.Link):\n\n    \"\"\"Batch normalization layer on outputs of linear or convolution functions.\n\n    This link wraps the :func:`~chainer.functions.batch_normalization` and\n    :func:`~chainer.functions.fixed_batch_normalization` functions.\n\n    It runs in three modes: training mode, fine-tuning mode, and testing mode.\n\n    In training mode, it normalizes the input by *batch statistics*. It also\n    maintains approximated population statistics by moving averages, which can\n    be used for instant evaluation in testing mode.\n\n    In fine-tuning mode, it accumulates the input to compute *population\n    statistics*. In order to correctly compute the population statistics, a\n    user must use this mode to feed mini batches running through whole training\n    dataset.\n\n    In testing mode, it uses pre-computed population statistics to normalize\n    the input variable. The population statistics is approximated if it is\n    computed by training mode, or accurate if it is correctly computed by\n    fine-tuning mode.\n\n    Args:\n        size (int or tuple of ints): Size (or shape) of channel\n            dimensions.\n        decay (float): Decay rate of moving average. It is used on training.\n        eps (float): Epsilon value for numerical stability.\n        dtype (numpy.dtype): Type to use in computing.\n\n    See: `Batch Normalization: Accelerating Deep Network Training by Reducing\\\n          Internal Covariate Shift <http://arxiv.org/abs/1502.03167>`_\n\n    .. seealso::\n       :func:`~chainer.functions.batch_normalization`,\n       :func:`~chainer.functions.fixed_batch_normalization`\n\n    Attributes:\n        gamma (~chainer.Variable): Scaling parameter.\n        beta (~chainer.Variable): Shifting parameter.\n        avg_mean (~chainer.Variable): Population mean.\n        avg_var (~chainer.Variable): Population variance.\n        N (int): Count of batches given for fine-tuning.\n        decay (float): Decay rate of moving average. It is used on training.\n        eps (float): Epsilon value for numerical stability. This value is added\n            to the batch variances.\n\n    \"\"\"\n    def __init__(self, size, decay=0.9, eps=1e-5, dtype=numpy.float32):\n        super(BatchNormalization, self).__init__()\n        self.add_param('gamma', size, dtype=dtype)\n        self.gamma.data.fill(1)\n        self.add_param('beta', size, dtype=dtype)\n        self.beta.data.fill(0)\n        self.add_persistent('avg_mean', numpy.zeros(size, dtype=dtype))\n        self.add_persistent('avg_var', numpy.zeros(size, dtype=dtype))\n        self.add_persistent('N', 0)\n        self.decay = decay\n        self.eps = eps\n\n    def __call__(self, x, test=False, finetune=False):\n        \"\"\"Invokes the forward propagation of BatchNormalization.\n\n        BatchNormalization accepts additional arguments, which controls three\n        different running mode.\n\n        Args:\n            x (Variable): An input variable.\n            test (bool): If ``True``, BatchNormalization runs in testing mode;\n                it normalizes the input using pre-computed statistics.\n            finetune (bool): If ``True``, BatchNormalization runs in\n                fine-tuning mode; it accumulates the input array to compute\n                population statistics for normalization, and normalizes the\n                input using batch statistics.\n\n        If ``test`` and ``finetune`` are both ``False``, then\n        BatchNormalization runs in training mode; it computes moving averages\n        of mean and variance for evaluation during training, and normalizes the\n        input using batch statistics.\n\n        \"\"\"\n        use_batch_mean = not test or finetune\n\n        if use_batch_mean:\n            func = batch_normalization.BatchNormalizationFunction(self.eps)\n            ret = func(x, self.gamma, self.beta)\n\n            if finetune:\n                self.N += 1\n                decay = 1. - 1. / self.N\n            else:\n                decay = self.decay\n\n            m = x.data.size // self.gamma.data.size\n            adjust = m / max(m - 1., 1.)  # unbiased estimation\n            self.avg_mean *= decay\n            func.mean *= 1 - decay  # reuse buffer as a temporary\n            self.avg_mean += func.mean\n            del func.mean\n            self.avg_var *= decay\n            func.var *= (1 - decay) * adjust  # reuse buffer as a temporary\n            self.avg_var += func.var\n            del func.var\n        else:\n            mean = variable.Variable(self.avg_mean, volatile='auto')\n            var = variable.Variable(self.avg_var, volatile='auto')\n            ret = batch_normalization.fixed_batch_normalization(\n                x, self.gamma, self.beta, mean, var, self.eps)\n        return ret\n\n    def start_finetuning(self):\n        \"\"\"Resets the population count for collecting population statistics.\n\n        This method can be skipped if it is the first time to use the\n        fine-tuning mode. Otherwise, this method should be called before\n        starting the fine-tuning mode again.\n\n        \"\"\"\n        self.N = 0\n", "path": "chainer/links/normalization/batch_normalization.py"}], "after_files": [{"content": "import numpy\n\nfrom chainer import cuda\nfrom chainer.functions.normalization import batch_normalization\nfrom chainer import link\nfrom chainer import variable\n\n\nclass BatchNormalization(link.Link):\n\n    \"\"\"Batch normalization layer on outputs of linear or convolution functions.\n\n    This link wraps the :func:`~chainer.functions.batch_normalization` and\n    :func:`~chainer.functions.fixed_batch_normalization` functions.\n\n    It runs in three modes: training mode, fine-tuning mode, and testing mode.\n\n    In training mode, it normalizes the input by *batch statistics*. It also\n    maintains approximated population statistics by moving averages, which can\n    be used for instant evaluation in testing mode.\n\n    In fine-tuning mode, it accumulates the input to compute *population\n    statistics*. In order to correctly compute the population statistics, a\n    user must use this mode to feed mini batches running through whole training\n    dataset.\n\n    In testing mode, it uses pre-computed population statistics to normalize\n    the input variable. The population statistics is approximated if it is\n    computed by training mode, or accurate if it is correctly computed by\n    fine-tuning mode.\n\n    Args:\n        size (int or tuple of ints): Size (or shape) of channel\n            dimensions.\n        decay (float): Decay rate of moving average. It is used on training.\n        eps (float): Epsilon value for numerical stability.\n        dtype (numpy.dtype): Type to use in computing.\n\n    See: `Batch Normalization: Accelerating Deep Network Training by Reducing\\\n          Internal Covariate Shift <http://arxiv.org/abs/1502.03167>`_\n\n    .. seealso::\n       :func:`~chainer.functions.batch_normalization`,\n       :func:`~chainer.functions.fixed_batch_normalization`\n\n    Attributes:\n        gamma (~chainer.Variable): Scaling parameter.\n        beta (~chainer.Variable): Shifting parameter.\n        avg_mean (~chainer.Variable): Population mean.\n        avg_var (~chainer.Variable): Population variance.\n        N (int): Count of batches given for fine-tuning.\n        decay (float): Decay rate of moving average. It is used on training.\n        eps (float): Epsilon value for numerical stability. This value is added\n            to the batch variances.\n\n    \"\"\"\n    def __init__(self, size, decay=0.9, eps=1e-5, dtype=numpy.float32):\n        super(BatchNormalization, self).__init__()\n        self.add_param('gamma', size, dtype=dtype)\n        self.gamma.data.fill(1)\n        self.add_param('beta', size, dtype=dtype)\n        self.beta.data.fill(0)\n        self.add_persistent('avg_mean', numpy.zeros(size, dtype=dtype))\n        self.add_persistent('avg_var', numpy.zeros(size, dtype=dtype))\n        self.add_persistent('N', 0)\n        self.decay = decay\n        self.eps = eps\n\n    def __call__(self, x, test=False, finetune=False):\n        \"\"\"Invokes the forward propagation of BatchNormalization.\n\n        BatchNormalization accepts additional arguments, which controls three\n        different running mode.\n\n        Args:\n            x (Variable): An input variable.\n            test (bool): If ``True``, BatchNormalization runs in testing mode;\n                it normalizes the input using pre-computed statistics.\n            finetune (bool): If ``True``, BatchNormalization runs in\n                fine-tuning mode; it accumulates the input array to compute\n                population statistics for normalization, and normalizes the\n                input using batch statistics.\n\n        If ``test`` and ``finetune`` are both ``False``, then\n        BatchNormalization runs in training mode; it computes moving averages\n        of mean and variance for evaluation during training, and normalizes the\n        input using batch statistics.\n\n        \"\"\"\n        use_batch_mean = not test or finetune\n\n        if use_batch_mean:\n            func = batch_normalization.BatchNormalizationFunction(self.eps)\n            ret = func(x, self.gamma, self.beta)\n\n            if finetune:\n                self.N += 1\n                decay = 1. - 1. / self.N\n            else:\n                decay = self.decay\n\n            with cuda.get_device(x.data):\n                m = x.data.size // self.gamma.data.size\n                adjust = m / max(m - 1., 1.)  # unbiased estimation\n                self.avg_mean *= decay\n                func.mean *= 1 - decay  # reuse buffer as a temporary\n                self.avg_mean += func.mean\n                del func.mean\n                self.avg_var *= decay\n                func.var *= (1 - decay) * adjust  # reuse buffer as a temporary\n                self.avg_var += func.var\n                del func.var\n        else:\n            mean = variable.Variable(self.avg_mean, volatile='auto')\n            var = variable.Variable(self.avg_var, volatile='auto')\n            ret = batch_normalization.fixed_batch_normalization(\n                x, self.gamma, self.beta, mean, var, self.eps)\n        return ret\n\n    def start_finetuning(self):\n        \"\"\"Resets the population count for collecting population statistics.\n\n        This method can be skipped if it is the first time to use the\n        fine-tuning mode. Otherwise, this method should be called before\n        starting the fine-tuning mode again.\n\n        \"\"\"\n        self.N = 0\n", "path": "chainer/links/normalization/batch_normalization.py"}]}