In the present paper we investigate the reproducibility of relative binding free energy calculations using three MD packages and two free energy estimators. Use of ensemble based simulations will be made to control uncertainties, as is essential for any calculation reliant on chaotic MD trajectories. Using ensembles to provide robust error control, we aim to identify statistically significant differences in the results from the different MD packages and estimators.