Can binary code such as an embedded OS or user application code be stored using silicon physical gates
In fact, there are hard-coded bits of data on many processors and other such devices, even complete bootloaders.

There are many ways to do that, sometimes it's just an array of hardwired shorts to VCC or GND that are selected in a grid, just like the bit-cells in an EEPROM device. 

Or sometimes it's a row driver with diodes and pull down or pull up resistors, or a similar trick, where the row driver decodes the address it's given, much like a 3-to-8 decoder, but larger:

Consider this "8 row, 3bit Hardwired rom":






simulate this circuit &ndash; Schematic created using CircuitLab

And since I'm not a chip designer, I'm probably overlooking some details here and there, but I'm quite sure there's a lot of ways this similar trick gets done. In some, or even many, cases they may make a grid with all diodes, or links, present and then laser etch away the paces where they want the opposite bit to come out, for example with unique IDs, where every chip gets lasered an increasing number in a fixed spot in the memory, that will not decay over time like Flash or EEPROM (though the latter has become nearly ageless by now).