Riga, had become Lutheran and was de jure dissolved, became again suffragan to an archdiocese. Ermland diocese, together with the new Berlin diocese and Schneidemühl prelature joined the new Eastern German Ecclesiastical Province () under the newly elevated Metropolitan Archbishop Adolf Bertram of Breslau. In 1932, Kaller consecrated the new diocesan seminary for priests in Braunsberg in East Prussia (today's Braniewo). Under his jurisdiction, Ermland diocese issued a new diocesan hymnal and a diocesan ritual (cf. Rituale Romanum) in Latin and the three native languages of the diocesan parishioners (German, Lithuanian and Polish). Kaller was also appointed apostolic visitator to the then 8,000 Catholic faithful in Memelland, a Lithuanian-annexed formerly East Prussian area, whose then four Catholic parishes had been seceded from Ermland diocese and subsequently formed part of the Territorial Prelature of Memel (Klaipėda)]], ; ; ) existing between 1926 and 1991. Kaller and other members of the German Catholic and Protestant Churches formulated their opposition to the policy of Nazi mysticism early on (cf. Struggle of the churches). German clergy who opposed Adolf Hitler or supported refugees were strongly persecuted under the Nazi dictatorship. On 10 June 1939, Pope Pius XII appointed Kaller apostolic administrator of the Territorial Prelature of Memel, after Lithuania had ceded Memelland under German pressure to Nazi Germany in March the same year. In 1942, Kaller applied to Nuncio Cesare Orsenigo to resign from episcopate in order to administer services at Theresienstadt, but his wish was not granted. On 7 February 1945, during World War II, the Nazi Schutzstaffel forced Kaller out of his episcopal office while the Soviet Red Army was overrunning Ermland diocese. Kaller had appointed Frauenburg's Cathedral dean Aloys Marquardt (1891–1972) as vicar general to the see. After World War II After World War I. most Germans were expelled to Allied-occupied Germany, including Marquardt who had to leave in July. Frauenburg's cathedral chapter then elected the aged Canon Johannes (Jan) Hanowski, a German of Polish ethnicity and long-term archpriest of Allenstein (today's Olsztyn), as capitular vicar, i.e. provisional head of the see, on 28 July 1945. Kaller, who had been stranded by the end of the war in Halle upon Saale, made his -long way back to his see and arrived on one of the first nights of August 1945 in Allenstein/Olsztyn, taking on the jurisdiction from Hanowski. He started to develop new plans for his diocese especially aiming at overcoming