

初めて利用します。  
私は、chainerを利用してConvolutionalLSTMという機構を用いた機械学習の研究を行っております。

MovingMnistを利用した実験ではプログラムが動いたのですが、手持ちのデータを利用しようとした所エラーが出てしまい、困っております。  
入力データは、320x256のbmpデータ(1チャンネル)になります。

**エラー内容**

[code]

    /home/denko/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
      from ._conv import register_converters as _register_converters
    
    Exception in main training loop: Unsupported dtype object
    Traceback (most recent call last):
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/training/trainer.py", line 299, in run
        update()
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/training/updater.py", line 223, in update
        self.update_core()
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/training/updater.py", line 228, in update_core
        in_arrays = self.converter(batch, self.device)
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/dataset/convert.py", line 93, in concat_examples
        [example[i] for example in batch], padding[i])))
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/dataset/convert.py", line 35, in to_device
        return cuda.to_gpu(x, device)
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/backends/cuda.py", line 275, in to_gpu
        return _array_to_gpu(array, device_, stream)
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/backends/cuda.py", line 322, in _array_to_gpu
        return cupy.asarray(array)
      File "/home/denko/anaconda3/lib/python3.6/site-packages/cupy/creation/from_data.py", line 61, in asarray
        return core.array(a, dtype, False)
      File "cupy/core/core.pyx", line 2070, in cupy.core.core.array
      File "cupy/core/core.pyx", line 2101, in cupy.core.core.array
    Will finalize trainer extensions and updater before reraising the exception.
    Traceback (most recent call last):
      File "train.py", line 81, in <module>
        train()
      File "train.py", line 69, in train
        trainer.run()
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/training/trainer.py", line 313, in run
        six.reraise(*sys.exc_info())
      File "/home/denko/anaconda3/lib/python3.6/site-packages/six.py", line 693, in reraise
        raise value
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/training/trainer.py", line 299, in run
        update()
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/training/updater.py", line 223, in update
        self.update_core()
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/training/updater.py", line 228, in update_core
        in_arrays = self.converter(batch, self.device)
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/dataset/convert.py", line 93, in concat_examples
        [example[i] for example in batch], padding[i])))
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/dataset/convert.py", line 35, in to_device
        return cuda.to_gpu(x, device)
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/backends/cuda.py", line 275, in to_gpu
        return _array_to_gpu(array, device_, stream)
      File "/home/denko/anaconda3/lib/python3.6/site-packages/chainer/backends/cuda.py", line 322, in _array_to_gpu
        return cupy.asarray(array)
      File "/home/denko/anaconda3/lib/python3.6/site-packages/cupy/creation/from_data.py", line 61, in asarray
        return core.array(a, dtype, False)
      File "cupy/core/core.pyx", line 2070, in cupy.core.core.array
      File "cupy/core/core.pyx", line 2101, in cupy.core.core.array
    ValueError: Unsupported dtype object
    
[/code]

**コード**

\--dataset_for_rr--

[code]

    import numpy as np
    import chainer
    from PIL import Image
    from chainer import cuda, Variable
    
    class rrDataset(chainer.dataset.DatasetMixin):
        def __init__(self,l,r,inf,outf):
            self.rr_list = np.load('/media/denko/OS/data/rr_list/test.npy')
            self.l = l
            self.r = r
            self.inf = inf
            self.outf = outf
            self.flames = self.inf + self.outf
            self.num = self.r - self.l
            self.xp = cuda.cupy
    
        def __len__(self):
            return self.num
    
        def get_example(self,i):
            self.ind = self.l + i
            self.data = np.empty((self.flames,1,256,320),dtype=np.float32)
    
            for j in range(self.flames):
                self.bmp_name = str(int(self.rr_list[self.ind][j]))
                self.year = self.bmp_name[0:4]
                self.month = self.bmp_name[4:6]
                self.path = '/media/denko/OS/data/bmp/' + self.year + '/' + self.month + '/' + self.bmp_name + '.bmp'
                self.bmp_data = Image.open(self.path)
                self.bmp_data = np.array(self.bmp_data).astype(dtype=np.float32)
                self.bmp_data = self.bmp_data / 255
                self.data[j,0,:,:] = self.bmp_data
    
            self.data = Variable(np.array(self.data,dtype=np.float32))
    
            return self.data[:self.inf,0,:,:], self.data[self.inf:self.flames,0,:,:]
    
[/code]


CC BY-SA 4.0
2018-09-25T06:19:02.180
48693
2018-09-26T01:22:47.620
2018-09-25T06:31:00.623
3060
30229
question
python chainer
Chainerで学習時にエラーが出てしまい、学習できない


短く言えば，

[code]

     34         self.data = Variable(np.array(self.data,dtype=np.float32))
    
[/code]

を

[code]

     34         self.data = np.array(self.data,dtype=np.float32)
    
[/code]

とすればよいでしょう．

以下は詳細です．  
わかりにくかったので行をつけさせていただきました．

[code]

      1 import numpy as np
      2 import chainer
      3 from PIL import Image
      4 from chainer import cuda, Variable
      5
      6 class rrDataset(chainer.dataset.DatasetMixin):
      7     def __init__(self,l,r,inf,outf):
      8         self.rr_list = np.load('/media/denko/OS/data/rr_list/test.npy')
      9         self.l = l
     10         self.r = r
     11         self.inf = inf
     12         self.outf = outf
     13         self.flames = self.inf + self.outf
     14         self.num = self.r - self.l
     15         self.xp = cuda.cupy
     16
     17     def __len__(self):
     18         return self.num
     19
     20     def get_example(self,i):
     21         self.ind = self.l + i
     22         self.data = np.empty((self.flames,1,256,320),dtype=np.float32)
     23
     24         for j in range(self.flames):
     25             self.bmp_name = str(int(self.rr_list[self.ind][j]))
     26             self.year = self.bmp_name[0:4]
     27             self.month = self.bmp_name[4:6]
     28             self.path = '/media/denko/OS/data/bmp/' + self.year + '/' + self.month + '/' + self.bmp_name + '.bmp'
     29             self.bmp_data = Image.open(self.path)
     30             self.bmp_data = np.array(self.bmp_data).astype(dtype=np.float32)
     31             self.bmp_data = self.bmp_data / 255
     32             self.data[j,0,:,:] = self.bmp_data
     33
     34         self.data = Variable(np.array(self.data,dtype=np.float32))
     35
     36         return self.data[:self.inf,0,:,:], self.data[self.inf:self.flames,0,:,:]
    
[/code]

問題はmetropolis氏が指摘しているように，34行目でVaribleを返しているところです．DatasetMixinは通常np.ndarrayまたはnp.ndarrayのタプルを返すことが想定されています．

検証用に短いコードを示します．

[code]

      1 from chainer.dataset.convert import concat_examples
      2 from chainer import Variable
      3 import numpy as np
      4
      5 data_np_1 = np.arange(9, dtype=np.float32).reshape((3, 3))
      6 data_np_2 = data_np_1 * 2
      7 data_Variable_1 = Variable(data_np_1)
      8 data_Variable_2 = Variable(data_np_2)
      9
     10 batch_1_np = (data_np_1[:1, :], data_np_1[1:, :])
     11 batch_2_np = (data_np_2[:1, :], data_np_2[1:, :])
     12
     13 batch_1_Variable = (data_Variable_1[:1, :], data_Variable_1[1:, :])
     14 batch_2_Variable = (data_Variable_2[:1, :], data_Variable_2[1:, :])
     15
     16 print(concat_examples([batch_1_np, batch_2_np]))
     17 print(concat_examples([batch_1_Variable, batch_2_Variable]))
     18
    
[/code]

結果が以下のようになります．

[code]

      (
      array([
              [[0., 1., 2.]],
              [[0., 2., 4.]]
            ], dtype=float32),
      array([
              [[ 3.,  4.,  5.], [ 6.,  7.,  8.]],
              [[ 6.,  8., 10.], [12., 14., 16.]]
            ], dtype=float32)
    )
    
    (
      array([
              [[variable(0.), variable(1.), variable(2.)]],
              [[variable(0.), variable(2.), variable(4.)]]
            ], dtype=object),
      array([
              [[variable(3.), variable(4.), variable(5.)], [variable(6.), variable(7.), variable(8.)]],
              [[variable(6.), variable(8.), variable(10.)], [variable(12.), variable(14.), variable(16.)]]
            ], dtype=object)
    )
    
[/code]

このように，dtypeがobjectであるndarrayが生成されているため，  
ValueError: Unsupported dtype object  
と言われるわけです．

これは`chainer.dataset.convert._concat_arrays`内で，

[code]

    if not isinstance(arrays[0], numpy.ndarray) and\
       not isinstance(arrays[0], cuda.ndarray):
        arrays = numpy.asarray(arrays)
    
[/code]

が呼ばれており，numpy.asarrayにVariableを渡すと，返り値が通常のndarrayにならないことが原因です．

以下はTipsなので，読み飛ばしていただいても結構です．

`inf`はmathでInfinityを表す予約語です．使わないことが推奨されます．  
同様に`.data`属性はnumpyとcupyでメモリへのポインタを表す予約語,
Chainerでxp.ndarrayを表す予約語なので，自分では使わないことが推奨されます．(実際にはChainerではv3.0.0以降予約語から解除されていますが，これはnumpy,
cupyとの予約語の競合を避けたためです)


CC BY-SA 4.0
2018-09-26T01:22:47.620
48711
2018-09-26T01:22:47.620
29620
48693
answer