How to convert mm to xy coordiantes in pcb xy files
The positive axes of an image coordinate system point right and down, while they point right and up in gerber, as you already wrote.

And an image has its origin in the upper left, while gerber also allows negative coordinates. I.e. the origin can be everywhere. Even worse, you typically know the width and height of an image, while a gerber file does not contain any information about the lowest and highest values in x and y.  

For example, this is a gerber file of two rectangles drawn in EAGLE, one right and above the origin, the other left and below:


G01*
X000000Y000000D02*
D10*
X000500Y001000D02*

  ###   draw first rectangle ###
X000500Y002000D01*
X001500Y002000D01*
X001500Y001000D01*
X000500Y001000D01*

X-001000Y-001000D02*

  ###   draw second rectangle with negative coords ###
X-001000Y-002000D01*
X-002000Y-002000D01*
X-002000Y-001000D01*
X-001000Y-001000D01*

X000000Y000000D02*
M02*


Your very first step is to find out the gerber coordinate range displayed in your image, or at least the lowest values for x and y. This can be tricky, since for example coordinates can be given relative to the last coordinate. 

Once you have the lowest values \$x_{low}\$ and \$y_{low}\$, you can convert given xy-coodinates  to pixel coordinates:

$$x_{pix}=(x-x_{low})\cdot U\cdot 600dpi$$
$$y_{pix}=ImageHeight - (y-y_{low})\cdot U\cdot 600dpi$$

\$U\$ is the unit conversion factor to inch, i.e. 0.001 if the gerber file uses mills, and 0.0393701 if it uses mm.

This of course assumes that the xy-values in your table are given in the gerber file coordinate system.



EDIT, after looking at your gerber files from the comments:

Here is a screenshot of your "game controller" PCB, drawn with gerbv:



The lower left corner of the boundary is somewhere at 1951/2900, far away from the origin! The coordinates displayed at the bottom are the coordinates of my mouse, pointing to the center of C18. They match well with the coordinates 2865/4040 from the  pick'n'place file. It's obvious that you have to adjust the coordinates of the parts to the offset.

You also have added an image of the board, which is not 600 dpi (at 1.25" board size, it must be about 750px wide).
Your image is rotated by 180°, which makes sense for the readability of the labels. But that's not what's in the file, and my formula have to be altered for this.

And the picture reveals that there are some pixels extra margin around the entire PCB, so the yellow lines don't touch the border. This has to be taken into account, too.



You added an other board, the "Nyx PCB". This one has the same orientation as my gerbv, making sense again. This board also has offsets in x and y, though they're quite tiny. And the image also has a border.