We have performed a systematic assessment of the computation of absolute binding free energies for protein-protein and protein-ligand systems by means of the umbrella sampling along a simple 1D reaction coordinate technique, currently one of the most popular techniques for calculating the ABFE. By demonstrating that independent repeats of US simulations can lead to significant differences in the PMF profiles, sometimes on the order of the actual ABFE, we revealed a lack of reproducibility that undermines the reliability of this approach in practical applications. We could attribute this lack of reproducibility to the inevitable differences between the pathways. We also demonstrate that for the barnase-barstar and trypsin-benzamidine systems, adaptive-biasing approaches, which sample multiple pathways in a single simulation, provide reproducible results and hence more reliable free energy estimates. For such approaches to become more widely used, however, further benchmarking will be needed to find the optimal parameters for running such simulations. We anticipate that our results will provide an incentive to (i) move away from US based techniques towards more robust methods (e.g. adaptive-biasing methods) for ABFE computation; and (ii) validate the approach chosen for the calculation by scrutinizing its accuracy, convergence, and