This study presents ∆∆G predictions using an open-source ensemble-simulation based alchemical free energy method for a large dataset of 503 diverse ligand pairs bound to 14 different targets covering a broad range of protein classes and 305 different ligands. Such a detailed and systematic analysis of the various factors affecting RBFE predictions enables us to make definitive recommendations for the implementation of relative free energy calculations.