What is the shape of conv3d and conv3d_transpose?
Although your input data is three dimensional, you have to use Conv2D for your task. I guess Conv3d is used for data with temporal characteristic, yours is just a simple picture. To illustrate why you should Conv2D, suppose your input image is 224 * 224 * 3 and you employ a Conv2D layer with 10 filters. You have to specify stride and padding in order to specify the output shape. You have to specify dimensions to illustrate the height and width of you filters, also known as kernels, filter size will affect the output size if you assign padding to 'valid'. Here, there is a point. Suppose you have specified the filter size a 10 * 10 filter, then if the input shape was 224 * 224 * 1, each filter would be of size 10 * 10 * 1 to fit the input area. Now that the input is of size 224 * 224 * 3 the size of each kernel is 10 * 10 * 3 to fit the input volume. Consider in all cases the output of each convolution operation, better to say cross correlation, is a scalar. For more information take a look at videos here and for your case I encourage you watching Convolution Over Volume.