I am trying to run the system as an appliance style device (no monitor, keyboard, etc.) To keep the OS protected, I have made the root file system read only and moved config files and mysql to a separate partition which is then sym-linked from the RO root directories.

The system boots up and runs fine for awhile, but after a random amount of time (anywhere from hours to days) it crashes and damages the file system so it won't boot again. The last one says that there is a bad magic number in the file system of the root partition.

My question then is, how can a read-only file system get corrupted? Is there any way the kernel can write to it still, or must there be some other electrical issue going on to cause the problem?