The buildings were n't damaged much during the Battle of Normandy as the German forces were defending Caen at the same time . The buildings in Bayeux were virtually untouched during the Battle of Normandy , the German forces being fully involved in defending Caen from the Allies .