1. A system for fitting a deformable model to an anatomical structure in a medical image to obtain a segmentation of the anatomical structure, comprising: an image interface for obtaining a first medical image of a patient and a second medical image of the patient, both medical images showing the anatomical structure and having been acquired by different medical imaging modalities or different medical imaging protocols, thereby establishing a different visual representation of the anatomical structure in both medical images; a processing subsystem configured for: i) fitting a first deformable model to the anatomical structure in the first medical image, thereby obtaining a first fitted model; ii) fitting a second deformable model to the anatomical structure in the second medical image, thereby obtaining a second fitted model wherein the second deformable model differs from the first deformable model for accommodating said different visual representation of the anatomical structure in both medical images; iii) mutually aligning the first fitted model and the second fitted model, and iv) after said mutual alignment, fusing the first fitted model and the second fitted model to obtain a fused model, wherein the processing subsystem is configured for mutually aligning the first fitted model and the second fitted model by: i) roughly aligning the first fitted model part of and the second fitted model, and ii) refining the alignment based on matching of pre-defined anchor structure.