 Charles and Zita temporarily resided at Castle Tata , the home of Count Esterházy , until a suitable permanent exile could be found . Malta was mooted as a possibility , but was declined by Lord Curzon , and French territory was ruled out due to the possibility of Zita 's brothers intriguing on Charles 's behalf . Eventually , the Portuguese island of Madeira was chosen . On 31 October 1921 , the former Imperial couple were taken by rail from Tihany to Baja , where the Royal Navy monitor Glow @-@ worm was waiting . They finally arrived at Funchal on 19 November . Their children were being looked after at Wartegg Castle in Switzerland by Charles 's step @-@ grandmother Maria Theresa , although Zita managed to see them in Zurich when her son Robert needed an operation for appendicitis . The children joined their parents in Madeira in February 1922 . 
