In this work, we propose a novel XAI approach tailored to networks built upon CDDD descriptors. This method, which we refer to as contextual explainability, is able to capture both atomic and structural contributions. We rely on explainability of concepts derived in the context of image analysis as well as on Img2Mol, a recently published optical molecular recognition model , that is able to translate images of molecules to their CDDD embeddings. We find that early layers in Img2Mol capture basic chemical features like atoms and bonds, while deeper layers learn more complex chemical structures, for instance, rings. By aggregating explanations from all the layers, we show that we can provide sparse and robust explanations that respect molecular symmetry and show both very localized highlights for particular atoms and more global importance for entire substructures.