AlwaysOn Manual failback not working correctly
I resolved this in the end. It was my own fault. I had connected using the name of the cluster "SQL-CLUSTER" rather than creating a listener. Originally I had tried to create a listener but was getting errors so stupidly I'd just ignored them. The errors were because I'd not set up the network properly (I'm a DBA not a network technician :) )

Anyway, after setting up the listener properly it works like a charm. Long story short... don't connect to the cluster, connect to the listener.