Here, we present the results of multiple repeats of umbrella sampling for the dissociation of protein-protein, protein-peptide, and protein-ligand complexes. We show that the PMF profiles obtained for each system differ by about 2-20 kcal/mol between individual repeats, indicating that the common procedure of relying on one repeat of US leads to unreliable and non-reproducible results. A similar issue was recently discussed for MD in general by Peter Coveney and co-authors, but here we focus specifically on US simulations. We demonstrate that very small differences in reference structures for the umbrellas can lead to large differences in the PMF profiles. Adaptive-biasing methods, such as AWH, rely on the sampling of multiple pathways in one simulation, and to understand if the inclusion of multiple pathways resolves the convergence issues, we also assess the performance of such simulation techniques for protein-protein and protein-ligand systems.