Configuration an FPGA on installation
The FPGA configuration itself is stored in SRAM (i.e. flip-flops). A newly started FPGA has an empty configuration where typically all pins are in a somewhat idle state (e.g. weak pull-up), and clocks are not forwarded inside the FPGA, so no activity takes place.

From that state, either the FPGA boots actively by accessing an external device (usually, flash memory), or waits until it is configured by external circuitry. Which road is taken usually depends on some resistors on some pins.

The internal block memory can also, but need not, be initialized from configuration data.

Shiny new boards are programmed like all others, by attaching to JTAG and either writing a configuration to the chip, or by poking the chip into writing to flash and subsequently triggering a reconfiguration so the data is loaded.