What would happen if I publish a modified version of an earlier block? Would it be rejected immediately by peer nodes?
Nodes would reject it immediately.

If you simply alter the data of B2 and send it, the block will be invalid by definition as the proof of work and merkle tree will not be valid anymore.

If you modify the data and rebuild the merkle tree and redo the proof of work to produce an alternative, valid B2' (note that everything else in this new block must also be valid, including transactions), nodes will still reject it as the chain going up to B4 has a greater amount of work.

Nodes will consider the chain with the greatest amount of total work as the valid chain. If you want to rewrite history, you will need to mine a new chain going back to the block you want to start rewriting from, and mine it to a point where its total work exceeds the current best chain. If you then broadcast this new chain of blocks, it will cause nodes to perform a reorganization, and drop the previous chain.

As for transactions returning to the mempool, that does happen in the event of a reorganization. All transactions from the dropped blocks are returned to the mempool. Usually, since miners generally pick similar transactions, the new blocks replacing the old ones will confirm mostly similar transactions, so the mempool will usually only be slightly different at the end of it.