Why would drush sql-sync fail to work? I have a development server set up in my drush aliases as @dev, and a UAT server set up as @uat.  I can successfully execute drush commands locally on both servers, and I can execute remote commands from UAT to the development server (e.g. drush @dev cc all or drush @dev upc or whatever).  However, when from the UAT server I execute

drush sql-sync @uat @dev


it appears to work, but when I first make a change on UAT and then sync and then check the same page on dev is has not changed.   I have tried drush @dev cc all afterward to make sure there is no caching problem, but this doesn't help. 

What could I be missing?  Or what could be failing?  How can I troubleshoot this?

UPDATE (corrected): If I log into @dev and execute the same command (in other words, having @dev pull data from @uat rather than having @uat push data to @dev) it works.   But I'd still like to get it working when executed from @uat.
There's not enough here to tell what's wrong, but there are a couple of things you can do to isolate the problem. Start with drush @uat sql-conf and drush @dev sql-conf; this is what Drush will use to look up the database records for the two sites.  If one of these fails to print the right information, that could cause sql-sync to fail.  Next, try drush sa --with-db @uat and again for @dev, and see if the database record matches what you saw with sql-conf.  If, for example, you copied a database record into your @dev alias and specified the wrong database (one different than the one specified in settings.php), then sql-sync might be successfully copying your production database to some other db on the dev machine.

If neither of these things work, you can try adding first -v (verbose output) and then -d (debug mode -- really a lot of output) and see if there is anything there that might help you find the problem.

Update: Did you make a typo in your update?  drush sql-sync @dev @uat pushes dev to uat, which seems like the opposite operation.  If drush sql-sync @uat @dev works on dev and not on uat, I would recommend carefully comparing your alias files on the two machines and see if there are any differences.  Especially check your 'databases' record in your alias definition, if you have any, and run the diagnostic tests I recommend above.  As a workaround, from uat you could run drush @dev sql-sync @uat @dev, which would remotely run drush sql-sync @uat @dev on the dev machine.