The name borough comes from of Bury , which has cognates in other Germanic languages such as the Old Norse `` borg '' meaning `` wall , castle '' ; and Gothic `` baurgs '' meaning `` city '' . The name Bury is etymologically connected with borough , which has cognates in other Germanic languages such as the Old Norse `` borg '' meaning `` wall , castle '' ; and Gothic `` baurgs '' meaning `` city '' .