Rolled back transactions during SQL Server Mirroring Failover
In FULL SAFETY mode you should be good to do this without data loss if there are no transactions in flight. Are you waiting for the state to change from Synchronizing to Synchronized after that restart? My guess is application users are in there when you are doing the failover, though.

If the applications are still connecting and data is still being modified then there are transactions in flight and that is the price you have to pay. Best to get a window and stop access. Shut down the site or apps accessing SQL, etc.