Citing a rejected paper in related work
Should I leave this citation in?


Definitely. A paper on arxiv is still related work regardless of its submission/acceptance status.


  Should I say that it has been rejected?


No. That seems inappropriate. The paper may soon be accepted elsewhere in which case your comment about rejection will be outdated.

Just cite it like you would any other paper. If you want, your bibliography could link to the openreview page where people could see the reviews and decision.


  Should the knowledge of the rejection change the way I talk about the paper?


Not really. Perhaps the reviews include some useful information. Otherwise treat it like any other paper.