One conceptually attractive approach to generative models are generative adversarial networks (GANs), which circumvent the need of explicitly evaluating a likelihood by adversarial training of two competing neural networks. GANs have been successfully applied to the generation of periodic structures. However, existing approaches have been based on global representations which means that the critic/decoder has to understand whole structures, of which there are many. As a result, the GANs generating periodic structures have required rather large amounts of training data on the order of 10k-1M structures. Focusing on learning the comparatively fewer physically significant local environments for evaluation could mitigate this data requirement. Recognizing viable compounds based on transferable local coordination or functional groups is fundamental to chemistry. The main challenge in producing stable and chemically accurate 3D structures using GANs lies in selecting the computational representation. The representation should continuously encode molecules and crystals while preserving periodicity, as well as rotational and translational symmetry, regardless of the number of atoms. In this paper we propose a "local GAN" (LoGAN), a GAN-based architecture which functions in a new, data-efficient manner. The architecture amalgamates the success of local structural descriptors used for MLFFs with the ability of the generator network to predict the whole structure in one step without requiring an evaluation of the PE or even local descriptors. Importantly, a prediction of the full structure in a single step allows our model to be equally successful for periodic and non-periodic systems. During training, our critic network utilizes the effectiveness of local descriptors, thus requiring a low amount of training data, and enabling the generation of structures at different stoichiometries from the training set. The rotational-and translational symmetries are taken care of by our choice of descriptors, while the permutational symmetry of similar atoms is handled by the critic model. We also include a programmable post-processing step which can be used to enforce various symmetries or boundary conditions of the goal systems. The LoGAN approach is tested on both on C 5 H 12 O isomers using only 39 similar molecules as training data and SiO 2 crystal structures using only 167 training samples, demonstrating its capability in a low-data regime.