In the image generation field, ControlNet 28 achieved success by leveraging the wellestablished encoding layers of large stable diffusion models, which were pre-trained with billions of images, to generate images under the guidance of input conditions. This approach allowed ControlNet to learn a diverse range of conditional models. Inspired by ControlNet, we introduce a novel diffusion model called Diff-Shape, which combines a pre-trained unconditional diffusion model with a graph control module for shape constrained 3D molecule generation. One unique feature of our method is that the pre-trained unconditional molecule generative model can be directly incorporated into Diff-Shape framework, therefore avoiding the computation cost of training from scratch. By taking in reference shape as point clouds with noise and partially obscured bond information, the graph control module guides diffusion process of molecular generation. To showcase the utility of Diff-Shape in drug design, several tasks including the shape-conditioned generation of chemically diverse molecular structures and the reference guided ligand generation in a structure-based drug design scenario were highlighted. In the former case, our results demonstrate that the Diff-Shape model significantly out-performed existing shape based generative models in terms of the shape similarity of the generation set to the reference shape, and, at the same time, can achieve balanced performance between high 3D shape similarity and relatively low 2D graph similarity which is the key requirement for identification of new scaffold. In the latter case, comparing with some binding pocket-based 3D generative models, our model can generate compounds with more favorable docking scores.