most of the men remained at a tavern on the outskirts of the town ; brown and one of the other men took me through the streets , accompanied by a silent tom christie , arriving at last at a large house of whitewashed brick .