In what situation can snapshot restore be blocked after SET SINGLE_USER?
When you set the database to single user, it allows a single user into the database.  If there's someone else trying to get into the database, and you don't have a connection to the database someone else could get into the database.
A single script like:
use master; 
alter database MyDb set single_user; 
use MyDb

...is still a single connection.  Something else would need to log into the SQL Server and into that database.  It could be a user, a job, an admin.
Setting the database into restricted user will block anyone who isn't a member of the db_owner fixed server role from connecting to the database.  I'd recommend that, or shutdown the app/web servers while you do the database restore.