Brown gained notice as the director of the Gouverneur Ambulatory Care Unit on Manhattan's Lower East Side.
In late 1967, Brown received a warning that an investigative reporter planned to expose homosexuals in the administration of Mayor John Lindsay and resigned rather than be forced out.