Batch Inserts and Batch Deletes
You can try this query to get good performance without using temp table or joining to delete from T1. And it's very simple.

SET ROWCOUNT 10000

WHILE EXISTS (SELECT TOP 1 'EXISTS' FROM T1 WHERE COLUMN=VALUE)
BEGIN
WAITFOR DELAY '00:00:10' -- You can pass it if you want interval between next delete
DELETE FROM T1
OUTPUT DELETED.* INTO T2
WHERE COLUMN=VALUE
END

SET ROWCOUNT 0


You can comment or remove WHERE clause if not required.

Thanks