Some of the remaining buildings from this period of its history include the Plumb House (now a museum open for tours seasonally) and the Coiner-Quesenbury House, built in 1806, believed to be the first brick house built in the town, which is still standing on Main Street.
Population growth to the town was slow at first.