PyTorch equivalent of `numpy.unpackbits`?
I am training a neural net on GPU. It uses a lot of binary input features.

Since moving data to/from GPU is expensive, I am looking for ways to make the initial representation more compact. Now, I encode my features as int8, move them over to GPU and then expand as float32:

# create int8
features = torch.zeros(*dims, dtype=torch.int8)

# fill in some data (set some features to 1.)
…

# move int8 to GPU
features = features.to(device=cuda, non_blocking=True)

# expand int8 as float32
features = features.to(dtype=float32)


Now, I am looking for ways to compress those binary features to bits instead of bytes.

NumPy has functions packbits and unpackbits

&gt;&gt;&gt; a = np.array([[2], [7], [23]], dtype=np.uint8)
&gt;&gt;&gt; b = np.unpackbits(a, axis=1)
&gt;&gt;&gt; b
array([[0, 0, 0, 0, 0, 0, 1, 0],
       [0, 0, 0, 0, 0, 1, 1, 1],
       [0, 0, 0, 1, 0, 1, 1, 1]], dtype=uint8)


Is there any way to unpack bits in PyTorch on GPU?

There is no similar functions at the time of writing this answer. However, a workaround is using torch.from_numpy as in: 

In[2]: import numpy as np
In[3]: a = np.array([[2], [7], [23]], dtype=np.uint8)
In[4]: b = np.unpackbits(a, axis=1)
In[5]: b
Out[5]: 
array([[0, 0, 0, 0, 0, 0, 1, 0],
       [0, 0, 0, 0, 0, 1, 1, 1],
       [0, 0, 0, 1, 0, 1, 1, 1]], dtype=uint8)
In[6]: import torch
In[7]: torch.from_numpy(b)
Out[7]: 
tensor([[0, 0, 0, 0, 0, 0, 1, 0],
        [0, 0, 0, 0, 0, 1, 1, 1],
        [0, 0, 0, 1, 0, 1, 1, 1]], dtype=torch.uint8)

