Swainby was the site where an old drover's road entered the North York Moors. The road continued south to Sutton Bank where it fed into the prehistoric road that is now the A170. Cattle from Ryedale were brought up to Swainby to be sold in the village.