he left the house at different times of the day , probably when his agenda dictated , and would usually , but not always , return for an afternoon session of spiritualism with a troupe of sect members .