How to pad a numpy 3D array (or torch tensor) with values from surrounding 3D arrays
I have a 3D numpy array of shape 3,3,3 to which I want to pad 2 layers of values from arrays surrounding it spatially, so that it becomes a 5,5,5 array.

What I have done so far using torch cat function (which works the same as numpy concat) to pad the y array, is the following:
x = torch.from_numpy(np.arange(1,28,1).reshape(3,3,3))
y = torch.from_numpy(np.arange(28,55,1).reshape(3,3,3))
z = torch.from_numpy(np.arange(55,82,1).reshape(3,3,3))

torch.cat((y,z[:,:2,:]), dim=1) #To concat z+ with 2 pads
torch.cat((x[:,1:,:],y), dim=1) #To concat z- with 2 pads

torch.cat((y,z[:,:,:2]), dim=2) #To concat x+ with 2 pads
torch.cat((x[:,1:,:],y), dim=1) #To concat x- with 2 pads

torch.cat((x,z[:2,:,:]), dim=2) #To concat y+ with 2 pads
torch.cat((x[1:,:,:],y), dim=1) #To concat y- with 2 pads

But it does not give me the right values. How can I acheive this?

If I understand correctly, what you want is not a regular array since each dimension have different range depending on the observed axis (i.e., it can't be geometrically represented as a cube - your picture is not a (n,n,n) array).
Anyways, in the somewhat lengthy following snippet, we create a (5,5,5) test 3D array from which the (3,3,3) array can be sampled. Then we consecutively concatenate to obtain the original array, after which we mask the unwanted cells so the output is what your picture shows. Note that you can replace logic operations with + or * when using boolean Numpy arrays.
import numpy as np

# Define dummy 3D field
n = 5
xx, yy, zz = np.ogrid[0:n, 0:n, 0:n]
field = np.sin(xx) + np.cos(yy) + np.tan(zz)

# Indices of 3 innermost elements - to form (3,3,3) array
i1, i2 = n//2 - 1, n//2 + 1
# Inner 3D array
subfield = np.copy(field)[i1:i2+1, i1:i2+1, i1:i2+1]

# Indices of &quot;inferior&quot; pads
x1 = y1 = z1 = np.arange(i1 - 1, i1)
# Indices of &quot;superior&quot; pads
x2 = y2 = z2 = np.arange(i2 + 1, i2 + 2)

# Padding in axis 0 (x)
padded = np.concatenate((field[x1, i1:i2+1, i1:i2+1], subfield))
padded = np.concatenate((padded, field[x2, i1:i2+1, i1:i2+1]))
# Padding in axis 1 (y)
padded = np.concatenate((field[i1-1:i2+2, y1, i1:i2+1], padded), axis = 1)
padded = np.concatenate((padded, field[i1-1:i2+2, y2, i1:i2+1]), axis = 1)
# Padding in axis 2 (z)
padded = np.concatenate((field[i1-1:i2+2, i1-1:i2+2, z1], padded), axis = 2)
padded = np.concatenate((padded, field[i1-1:i2+2, i1-1:i2+2, z2]), axis = 2)

# Check padded array is equal to original 3D array
print(np.all(padded == field))

## We now mask unwanted cells
indices = np.indices(padded.shape)
idx1, idx2 = indices == 0, indices == n - 1

xi, xf = idx1[0], idx2[0]
yi, yf = idx1[1], idx2[1]
zi, zf = idx1[2], idx2[2]

# Logical operations to mask proper slices
xm = (xi + xf) * (yi + yf + zi + zf)            # masking in axis 0
ym = (yi + yf) * (xi + xf + zi + zf)            # masking in axis 1
zm = (zi + zf) * (yi + yf + xi + xf)            # masking in axis 2

mask = xm + ym + zm
# Masked (5,5,5) array
masked_padded = np.ma.masked_where(mask, padded)

By the way, there must be more elegant ways to achieve the same result, but I have not used Numpy's advanced indexing that much :P
