Recognition of small objects
There is a big difference between the 2 datasets. When you train on the GTSRB dataset, your model performs image recognition, while for the GTSDB dataset you need to perform object detection. These are 2 related but different tasks (with detection being considerably harder).

You can consider using a sliding window approach, or use some prior attention model that will learn to extract the area of the signs for later classification.

I don't think that it is recommended to down sample images of 1360x800 pixels to 224x224.
(224x224)/(1360x800)=0.046. You are loosing 96% of the information (including your street signs which might reduced to nothing), granted most of it is not interesting, but simple down-sampling doesn't get to choose what to forfeit.