Will SQL Server check FK restrictions between row deletes in a single DELETE statement?
SQL Server only seems to be checking the FK constraint after all rows to be deleted in a single DELETE statement have been deleted, rather than in between each row deletion. Can I rely on this behaviour?

Yes.  Constraints are checked for the statement, not for each row, and not at the end of a transaction.
Here's the query plan for a DELETE from that table.

The delete occurs on the top branch, but the IDs of the deleted rows are spooled, and after the clustered index delete step, the spooled IDs are used to look up any rows that reference the deleted rows.  If any are found the DELETE will be rolled back (DML statements always run in a nested transaction internally), and a FK error raised.