Beyond the challenge of representing nanomaterials, it is also challenging to generate datasets sufficiently large to train DL models that can accurately predict the properties of heterogeneous, multi-component nanostructures. Although highthroughput experimental and computational approaches are growing in their availability and utility, the synthesis and simulation of complex heterostructures is often time-consuming, limiting the scale of available datasets and constraining campaigns to the "small data" regime where DL techniques often struggle. Modern DL models can also have difficulty extrapolating outside of the envelope of their training data, which is necessary for the discovery of novel materials with enhanced properties. Finally, the discovery of fundamentally new materials is complicated by the rough response surfaces of material properties with respect to their composition, necessitating dense and tedious "needle-in-a-haystack" searches across a parameter space. The prediction of materials with targeted properties, or inverse design, would be significantly accelerated by surrogate models that are differentiable so that gradient-based optimization techniques can be used to direct efficient searches. Thus, DL-guided inverse design of complex materials would benefit from the development of large structure-property datasets of complex nanomaterials, new methods to represent them across lengths scales, and differentiable models that are accurate out of their training distribution.