now was not the time or place for it , of course , but the memory of other times and other places made him shift , twisting away from her , not to trouble her with the bodily evidence of his thoughts .