The prevalence of unknown constraints in optimization problems across the natural sciences cannot be understated. In many molecular design tasks, candidates are selected from a virtual library of molecules which are thought to be synthetically accessible. Selection is followed by a synthesize-measure procedure, in which an attempt is made to synthesize the molecule, and if successful, its objective is determined. The synthesis step in this procedure could fail for several reasons (e.g. insufficient yield to make the property measurement, insufficient stability of the product, etc.). Although several approaches to computational prediction of organic reaction outcome have been described, these tools are typically regarded as coarse-grained proxies for synthetic accessibility. Therefore, synthetic accessibility of molecular candidates is often an unknown constraint which is ideally inferred on-the-fly for a novel design campaign. Experimental failures are not unique to organic synthesis. For example, Wakabayashi et al. studied molecular beam epitaxy of SrRuO 3 , a widely used metallic electrode in oxide electronics, in which experimental failures occur when the SrRuO 3 phase did not form, preventing measurement of the resistivity objective. While several approaches have been suggested for BO with unknown constraints, relatively little effort has been dedicated to evaluating the suitability of these strategies on representative optimization problems in the experimental sciences. Furthermore, we have found that a comprehensive optimization benchmark of all relevant strategies is missing. Finally, we found that many popular BO packages do not consider unknown constraints altogether, or implement only the simplest approaches to dealing with them. Our goal for this work is to provide practitioners of autonomous science driven by BO comprehensive insight into which strategies for dealing with unknown constraints work best in various scenarios. To this end, we implement and test several different approaches to dealing with unknown constraints in our research group's open-source BO Python package, Atlas. The remainder of this work is organized as follows. Sec. II formally introduces optimization with unknown constraints and compares and contrasts our problem to related work. Sec. III describes our approaches to learning the constraint function, our feasibility-aware acquisition functions, as well as limitations of our approach. Sec. IV.A presents a detailed comparison of optimization performance of each approach on synthetic benchmark surfaces. Finally, Sec. IV.B and IV.C detail two realworld applications: the inverse design of hybrid organicinorganic halide perovskite materials with stability constraints, and the discovery of BCR-Abl kinase inhibitors with synthetic accessibility constraints.