How to use an FPGA on your own PCB
It's the same story as with normal micro-controllers. FPGAs as well have a JTAG programming interface and the manufacturers offer the appropriate programmers. 

The difference to the normal development boards is, that they usually have a already built-in programmer, as you noticed (the USB interface). So you can either replicate that programmer on your PCB, what's expensive and makes your design complex so usually it's not recommendable or you place a header that fits the programmer.

Whether the image is volatile or not depends on how you program the chip. Most of the chips have a configurable bootloader that either gets the image from the internal or from an external flash, these are the persistent variants or the FPGA waits for a programming sequence. This can be the operator programming it or you can have another chip on your board that does that, so these are the volatile options. That of course only applies to the re-programmable chips.

You find programming information in the FPGA's User Guide