 This phase I trial studies the side effects and the best dose of Z-endoxifen hydrochloride in treating patients with estrogen receptor-positive ER breast cancer that has spread to other places in the body metastatic or has come back at or near the same place as the original tumor locally recurrent. Estrogen can cause the growth of breast cancer cells. Hormone therapy using Z-endoxifen hydrochloride may fight breast cancer by blocking the use of estrogen by tumor cells. 