Handling dimensions for RGB data with Keras CNN I'm trying to make work the code from Keras' documentation getting started. There is something I do not understand about handling RGB data. I made one work with MNIST data (which is greyscale), but I can't seem to figure out the CIFAR10.
HEIGHT = 200
WIDTH = 200
def build_model():


    inputs = keras.Input(shape=(HEIGHT, WIDTH, 3))

    # making things simple, I don't CenterCrop, just set the input to whatever I'm feeding it
    # I do rescale to 0-1 values
    x = Rescaling(scale=1.0 / 255)(inputs)

    # this basically the doc's architecture
    x = layers.Conv2D(filters=32, kernel_size=(3, 3), activation=&quot;relu&quot;, )(x)
    x = layers.MaxPooling2D(pool_size=(2, 2))(x)
    x = layers.Conv2D(filters=32, kernel_size=(3, 3), activation=&quot;relu&quot;,)(x)
    x = layers.MaxPooling2D(pool_size=(2, 2))(x)
    x = layers.Conv2D(filters=32, kernel_size=(3, 3), activation=&quot;relu&quot;,)(x)
    x = layers.GlobalAveragePooling2D()(x)
    outputs = layers.Dense(10, activation=&quot;softmax&quot;)(x)
    model = keras.Model(inputs=inputs, outputs=outputs)
    return model


# fake data, easier to play with shapes  than with the actual CIFAR10 data to debug
# shape is (500, 200, 200, 3), cifar10's is (60000, 32, 32, 3)
data = np.random.randint(0, 255, size=(500, HEIGHT, WIDTH, 3)).astype(&quot;float32&quot;)
# shape is (500,1) cifar10's is (60000,1). Just 10 categories to match the output layer
labels = np.random.randint(0,9, size=(500,1)).astype(&quot;int8&quot;)

print(&quot;got fake data... &quot;)
model = build_model()
print(&quot;model built... &quot;)
model.summary()
model.compile(optimizer=keras.optimizers.RMSprop(learning_rate=1e-3),
              loss=keras.losses.CategoricalCrossentropy())
print(&quot;model compile...&quot;)
model.fit(data, labels)
print(&quot;done&quot;)

For some reason, I get:
ValueError: Shapes (None, 1) and (None, 10) are incompatible

Strangely, if change the label's fake data to size=(500, 10) then it &quot;works&quot;. But obviously that makes no sense, since it would mean I have 10 labels for each sample. The MNIST dataset that works which I refer to can also be found in the linked documentation. The architecture of the network is much simpler (only Dense layers linked together).
What am I missing here? Why would it be so different for a colored image than it is for a greyscale?
I also tried the work with the data from     train, test = cifar10.load_data(), but it yields the same results.
Strangely, if change the label's fake data to size=(500, 10) then it &quot;works&quot;. But obviously that makes no sense, since it would mean I have 10 labels for each sample.

It would most certainly not mean that; it simple means that, for the type of softmax classification you are attempting here with CategoricalCrossentropy loss, your labels should be one-hot encoded, and not single digits, as here (hence the error).
You should

Either convert your single-digit labels to one-hot encoded ones using to_categorical (docs)
Or change your loss to sparse categorical cross-entropy (docs)

Changes in your code for the first approach:
labels_ohe = tf.keras.utils.to_categorical(labels, num_classes=10)
# [...]
model.fit(data, labels_ohe)

Changes in your code for the second approach:
model.compile(optimizer=keras.optimizers.RMSprop(learning_rate=1e-3),
              loss=keras.losses.SparseCategoricalCrossentropy())


The MNIST dataset that works which I refer to can also be found in the linked documentation.

The MNIST labels, like the CIFAR10 ones, are also single-difit integers and not one-hot encoded; your MNIST model from the example you refer to you works because of the loss=&quot;sparse_categorical_crossentropy&quot; setting.