Is the term page replacement actually a misnomer?
Is the term page replacement actually a misnomer?


No. 


A page is the entity in virtual memory.
A page frame is the real memory into which it is mapped.


So it is the page, not the frame, that is replaced. 

The metaphor is that the (virtual) page is the picture around which the frame is placed. You move the frame to picture different pages. See here.