Applications in clean energy, advanced manufacturing, biomedicine, photonics, and microelectronics increasingly demand new materials with complex structures and heterogeneous composition. In principle, machine learning (ML) offers a strategy to accelerate the discovery of such materials, as it has emerged as a transformational tool for the design of small molecules, bulk inorganic crystals, and even single-component nanomaterials. Deep learning (DL) approaches are particularly well suited to model the behavior of systems with large numbers of parameters, but several obstacles hinder DL from being used to guide the discovery of the complex materials often needed for real-world applications, including nanostructures and composites.