Deep learning test loss curve won't go down I've been working with Deep Learning projects for this current project that I am working on and it's basically a time series classification problem. Where given an array of time series data I need to classify the customers as being either honest or dishonest.
The current model that I have now only uses CNNs, but I'm planning on expanding it with LSTMs or other models in the future. Here is the code for my model.
model = Sequential([
    Input(batch_input_shape = (None, 1036, 1)),

    Conv1D(
        filters=32,
        kernel_size=3,
        padding='same',
        activation='relu',
        activity_regularizer=l2(5e-4),
    ),
    Conv1D(
        filters=16,
        kernel_size=3,
        padding='same',
        activation='relu',
        activity_regularizer=l2(5e-4),
    ),
    MaxPooling1D(),
    
    Conv1D(
        filters=8,
        kernel_size=3,
        padding='same',
        activation='relu',
        activity_regularizer=l2(5e-4),
    ),
    Conv1D(
        filters=8,
        kernel_size=3,
        padding='same',
        activation='relu',
        activity_regularizer=l2(5e-4),
    ),
    MaxPooling1D(),
    Flatten(),

    Dense(10, activation='relu'),
    Dropout(0.25),
    Dense(1, activation='sigmoid'),
])

model.compile(
    # optimizer=tf.keras.optimizers.Adam(learning_rate=1e-4),
    loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
    metrics=[f1_m,precision_m, recall_m, matthews_correlation, 'accuracy', fpr_m]
)

After training the model for 100 EPOCHs the loss curve is this

I've tried a number of things which are:

Reducing the network size. This still results in the same issue but at a different point in the network
Reducing the learning rate of the optimizer. Same with the point above. This seems to work but it is in fact changing when this happens.

I am open to any input or recommendations I could get on how to make the testing curve follow the training curve more. I should note that my dataset is imbalanced and I'm ONLY balancing the x_train and y_train using SMOTE and not making and balancing to the testing and validation datasets to keep the data as clean as possible.
Your learning curves suggest overfitting.
In this respect you should try

increasing dropout
adjust l1/l2 regularisation to higher levels
inspect class ratios in training and validation tests (&quot;learning on easy predicting on difficult&quot; imbalances)

Additionally, you may want to

add batchnorm layer
try different activations
check to what extend weights/bias of each layer get updated during training iterations, in case of vahishing/exploding gradients