there were only two or three years when it might have been possible , between his familys moving to the house on the ridge and his own going on from primary to grammar school on the other side of the town .