SQL Anywhere 12 - exporting data to Excel
The OUTPUT USING format of the statement is using a ODBC driver to create/send the data to.

This means you must have a ODBC driver on the server which can create XLS files.

There is a MS Office 2007 ODBC download available from microsoft which provides such a driver.

I don't know if there exist 64-Bit drivers (I think it must match with your dbserver 32/64bit) nor if Office 2010/2013/2016 ODBC drivers exist.

Docu output-statement