Promoting warm standby to master with Postgresql 9.4 using streaming replication
Solved it. The issue was that I had Monit running in the background which was picking up the backup database before I had written recovery.conf (the servers are being provisioned by chef).

The solution was to pass the -R flag to pg_basebackup in order to ensure the database was never started outside of recovery mode which caused it to be incompatible with the WALs and streaming replication from the master.