Why is GPU memory fixed?
Is there any fundamental reason (ie, electronic engineering reason, not marketing) why GPUs don't come in different amounts of memories for a specific GPU model? 


Don't know whether you mean that with "marketing reason" but: memory slots, the boards and auxillary electronics of exchangeable RAM "modules" cost money. That's the main reason why you don't do that.

Technically, RAM chips can behave quite differently, and that's why you have the SPD information on the EEPROM of your PC RAM module: the memory controller needs to know how to even use the specific RAM you just plugged in.

That flexibility of course makes the whole thing both more prone to errors, and complex – not something you want, again, on cost level.

Don't forget that restricting the number of possible RAM configurations (as in amount, and sizes of individual RAM ICs) also simplifies not only the memory interface, but the system architecture and probably even reduces die size of the actual processing units: if you know how much memory you have at which physical addresses, you don't have to even think about designing in e.g. address remapping units that are flexible.

Then: When you add a connector to the mix, the whole host-RAM connection, which, especially in graphic cards, is a very high-rate bus, gets significantly less deterministic than when you solder the memory to the same high-quality multi-layer board. That means that your memory controller now needs to do way complex things, like effectively equalizing the received signal from the RAM, and that drives up complexity, too. In fact, it will shift the development effort of "make damn sure your RAM traces are within specs, dear graphic cards designer" towards "make damn sure the integrated memory controller can deal with all sorts of absurd channel effects, dear GPU manufacturer"; and you can see how that is non-desirable to companies that form an duopol on high-end GPUs.


  or example, my place of work recently acquired a number of GPGPU servers. Each of the GPU's comes with I think 24 GB or RAM, but the FDTD simulations they run don't need anywhere near that amount of RAM


Yep, but exactly in the market of datacenter / HPC things, where hardware cost is less of an issue than reliability and extensibility, Nvidia GPUs with interfaces to "standard" DDR RAM modules exist. Nvidia themselves has the DGX-1 rack server.