Can you combine database mirroring with AlwaysOn Availability Groups?
You are correct, it is impossible to mirror a database that is part of an Always On Availability group.  You can consider logshipping.

Ref:

https://blogs.msdn.microsoft.com/sqlalwayson/2010/12/08/alwayson-availability-group-and-database-mirroring/


  "Does AlwaysOn Availability Group replace Database Mirroring going
  forward?” The short answer is we recommend that you migrate from the
  mirroring configuration or even mirroring and log shipping
  configuration to using Availability Group. Database Mirroring will
  still be available in the Denali release but will be phased out over
  subsequent releases. Log Shipping will continue to be available in
  future releases.


For the error you are getting read this document especially section: Windows System Requirements and Recommendations. 

https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/prereqs-restrictions-recommendations-always-on-availability#PrerequisitesForAGs