There were three hotels in Williamstown by 1841 and most of the men worked at the port . There were three hotels in Williamstown by 1841 and most of the men worked at occupations related to the port .