Ola Hallengren SQL Server Backup - CleanupTime
There's a separate @CleanupMode parameter that controls whether cleanup happens before or after. The default behavior is to do the cleanup AFTER. This is the safest option, but requires the most disk space. In the event of a failed backup, nothing is cleaned up, ensuring you have extra files, rather than having deleted a file without making a new one.

This is particularly important if backups were to fail repeatedly (cleaning up, but never making a new one) or when there's a very short retention. 

From the documentation:

CleanupMode

Specify if old backup files should be deleted before or after the backup has been performed.

Values:


BEFORE_BACKUP - Delete old backup files before the backup has been performed.
AFTER_BACKUP - Delete old backup files after the backup and verify has been performed. If the backup or verify failed, then no backup files are deleted. This is the default.