Would SQL Server in a Cluster failover if it is running out of RAM
Starting with SQL Server 2012 the cluster will have a more comprehensive health check which allows for a possible resource based health condition using the SP_Server_diagnostics procedure. https://msdn.microsoft.com/en-us/library/ff878233.aspx

More on that failover policy can be read about here: https://msdn.microsoft.com/en-us/library/ff878664.aspx#monitor

Unfortunately though SQL Server 2008R2 and earlier doesn't have much of a sophisticated cluster health check for the SQL Service. Simply the health check will just issue the following command "SELECT @@Servername" and if it returns in a timely fashion the health check passes. 

I would look instead though at why is the server under memory pressure and setting the max memory setting in the SQL Server instance putting a cap on that memory foot print.