xp_cmdshell in a transaction
No, absolutely impossible. Each sqlcmd invocation is a different session and transaction cannot span sessions.

Your use of a transaction on the script that invokes the sqlcmd is completely useless, it has nothing to do with the scripts being run.