Which filesystems get overwritten when applying a ROM? I'd like to better understand the process of writing a ROM (a zip package) to my Android phone.

Checking with adb shell, the following filesystems are mounted on my phone:

rootfs / rootfs ro,relatime 0 0

none /acct cgroup rw,relatime,cpuacct 0 0
none /sys/fs/cgroup tmpfs rw,seclabel,relatime,mode=750,gid=1000 0 0
none /dev/cpuctl cgroup rw,relatime,cpu 0 0

devpts /dev/pts devpts rw,seclabel,relatime,mode=600 0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,seclabel,relatime 0 0
selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0

/sys/kernel/debug /sys/kernel/debug debugfs rw,relatime 0 0
/dev/fuse /mnt/shell/emulated fuse rw,nosuid,nodev,relatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0

tmpfs /dev tmpfs rw,seclabel,nosuid,relatime,mode=755 0 0
tmpfs /mnt/asec tmpfs rw,seclabel,relatime,mode=755,gid=1000 0 0
tmpfs /mnt/obb tmpfs rw,seclabel,relatime,mode=755,gid=1000 0 0
tmpfs /mnt/fuse tmpfs rw,seclabel,relatime,mode=775,gid=1000 0 0

/dev/block/mmcblk0p3 /efs ext4 rw,seclabel,nosuid,nodev,noatime,user_xattr,barrier=1,journal_async_commit,data=ordered 0 0
/dev/block/mmcblk0p9 /system ext4 ro,seclabel,noatime,user_xattr,barrier=1,data=ordered 0 0
/dev/block/mmcblk0p8 /cache ext4 rw,seclabel,nosuid,nodev,noatime,errors=panic,user_xattr,barrier=1,journal_async_commit,data=ordered 0 0
/dev/block/mmcblk0p10 /preload ext4 rw,seclabel,nosuid,nodev,noatime,user_xattr,barrier=1,journal_async_commit,data=ordered 0 0
/dev/block/mmcblk0p12 /data ext4 rw,seclabel,nosuid,nodev,noatime,user_xattr,barrier=1,journal_async_commit,data=ordered,noauto_da_alloc,discard 0 0


Now, to the best of my knowledge, only the last five actually correspond to non-volatile memory. Could you explain what's the structure of such a zip package, and which devices are overwritten when a ROM update occurs through ODIN or CWM recovery?
As Izzy has said, this entirely depends on the device and the particular image you are flashing.  Some might flash a new recovery, some not.  Some might flash only the system partition and leave your data alone.  There are even packages that will flash a new bootloader.

If you want to see what a ROM contains, you should open it up and have a look.  Here is a basic explanation of the structure of a kernel flash package.