With the growing interest in automated procedures for the exploration of chemical space, it becomes more than ever pertinent to develop a robust, chemically meaningful ansatz for the definition of Z-matrices. In this work, we provide such an algorithm, overcoming the pitfalls commonly encountered when converting from the cartesian to the Z-matrix coordinate space and backwards. Analytic transformations of gradients from cartesian to Z-matrix coordinates are derived for the first time, improving on existing numerical algorithms for this task. In combination with symbolic algebra packages, the analytic gradients can be used for non-linearly constrained geometry optimisations in Z-matrix space.