Specifying neural network output layout for object detection
I'd say how you order your sequences or normalize data will have much less importance than where you get features for those classifier/detection vectors (where you attach those to your base CNN network).

Your idea falls into approach called "Single Shot Detector" (as opposed to region proposal/sliding windows) and I'd recommend looking at SSD (Liu et al) and YOLO (Redmon, Farhadi) models – there is much more to it to get it performing well.