What does the co-ordinate output in the yolo algorithm represent? My question is similar to this topic.
I was watching this lecture on bounding box prediction by Andrew Ng when I started thinking about output of yolo algorithm. 
Let's consider this example, We use 19x19 grids and only one receptive field with 2 classes, so our output will be => 19x19x1x5.
The last dimension(array of size 5) represents the following:

1) The class (0 or 1)
2) X-coordinate
3) Y-coordinate
4) height of the bounding box
5) Width of the bounding box


I don't understand whether X,Y coordinates represent the bounding box with respect to the size of entire image or just and receptive field(filter). In the video the bounding box is represented as a part of receptive field but logically receptive field is much smaller than bounding box and also people might tinker with filter size, so positioning bounding boxes with respect to filter makes no sense.

So, basically what does the coordinates of bounding boxes of an image represent ?
Simply the left high most vertex of the bounding box. 

With X,Y and it's height and width you can find the bounding box in the image. (With respect to the size of entire image)