Is there a way to limit the memory buffer pool for a database?
Is there a way I can limit AltDB to only use a maximum of 10GB. 


No, not even with Resource Governor. 

From BOL 


  The Database Engine supports a performance optimization mechanism called read-ahead. Read-ahead anticipates the data and index pages needed to fulfill a query execution plan and brings the pages into the buffer cache before they are actually used by the query. 


A good article to read : The clock hands of the buffer cache


  My goal is that more memory is used by MainDB.


Another alternative is to have MainDB on a different instance capped by Max Memory. This way you can reduce the max memory for AltDB.