In 1865 William's cousin, John W. Carter, joined the enterprise and the name became "Carter Bros. & Company."
The entire firm and both of its divisions and their separate buildings were destroyed the night of November 9, 1872, in what has been called the Great Boston Fire of 1872.