Here we present a 3D fragment-based reinforcement learning approach for the generation of novel molecules with optimized properties, called "DeepFMPO v3D". We furthermore developed a python package, ESP-Sim, for calculating molecular shape and electrostatic similarities. We benchmarked ESP-Sim on a variety of tasks including the evaluation of detailed 3D similarities, protein-ligand docking and re-scoring of docked ligands and report competitive performances. The inclusion of ESP scores into DeepFMPO promotes the generation of compounds similar to existing lead molecules, towards desirable sweet spot properties. The proposed method allows the calculation of high-quality partial charges (e.g., RESP with B3LYP/6-31G**) obtained using the quantum chemistry program Psi4. In a scaffold-hopping case-study we show that our approach of using shape and electrostatics similarities performs well. DeepFMPO v3D ranks known equipotent scaffolds higher and generates them earlier (i.e., speedier). The way the 3D method is implemented makes the approach essentially alignment-independent (on a molecular level) and does not require knowing the bioactive conformation. Both DeepFMPO v3D and ESP-Sim are freely available online.