1. A system for fitting a deformable model to an anatomical structure in a medical image to obtain a segmentation of the anatomical structure, comprising: an image interface for obtaining a first medical image of a patient and a second medical image of the patient, both medical images showing the anatomical structure and having been acquired by different medical imaging modalities or different medical imaging protocols, thereby establishing a different visual representation of the anatomical structure in both medical images; a processing subsystem configured for: i) fitting a first deformable model to the anatomical structure in the first medical image, thereby obtaining a first fitted model; ii) fitting a second deformable model to the anatomical structure in the second medical image, thereby obtaining a second fitted model; wherein the second deformable model differs from the first deformable model for accommodating said different visual representation of the anatomical structure in both medical images; iii) mutually aligning the first fitted model and the second fitted model, and iv) after said mutual alignment, fusing the first fitted model and the second fitted model to obtain a fused model, wherein the processing subsystem is configured for mutually aligning the first fitted model and the second fitted model by: i) roughly aligning the first fitted model and the second fitted model, and ii) refining the alignment based on matching of pre-defined anchor structures depicted in both medical images, the anchor structures being subsets of the medical images, wherein a deformation is calculated to match the anchor structure of the second fitted model to the anchor structure of the first fitted model, and wherein the calculated anchor structure deformation is applied to the second fitted model in its entirety.