Review claims reference was omitted, but it was not
The manuscript was rejected and I do not plan to resubmit

Then do not respond.

Should I accept this and move on

Yes.  Reviewers make mistakes.  That's normal.
Most people who think they know who the reviewer was are wrong.