How is the size of a block calculated? How is the blocksize currently calculated in Bitcoin Core? I'm specifically asking about how it's done for the purposes of enforcing the 1MB limit. If it's done by calculating the serialization length, what serialization format is used?
The block size is the combination of the block header and the list of transactions. Specifically, the block header has these fields:


version - 4 bytes
previous block header hash - 32 bytes
merkle root hash - 32 bytes
time - 4 bytes
nBits (encoded POW target) - 4 bytes
nonce - 4 bytes


So the total bytes for a block header is
4 + 32 + 32 + 4 + 4 + 4 = 80 bytes

After the block header is the list of transactions which is


transaction count - 1-9 bytes
transactions - varies


Transactions can vary greatly in terms of the number of bytes, but the general serialization of a single transaction is:


version - 4 bytes
tx in size - 1-9 bytes
tx ins - see below
tx out size - 1-9 bytes
tx outs - see below
locktime - 4 bytes


Each tx in has these fields:


previous tx out hash - 32 bytes
previous tx out position - 4 bytes
pubkey script - varies depending on script, max 10,000 bytes
signature script - varies
sequence - 4 bytes


Each tx out has these fields:


value - 8 bytes
pubkey script length - 1-9 bytes
pubkey script - varies


The sum total of all these things in a block determine the size, which currently has to be less than 1 MB.