How would a CPU know if a piece of data is in cache or RAM or on disk?
How does the CPU know if a piece of data is in L1 cache?  The L1 cache is part of the CPU (it's on the same chip) so the CPU just asks the L1 cache "do you have the data for address X?"; if it gets the answer "yes", it is good; if it gets the answer "no", then the CPU tries to read from RAM.

How does the CPU know if a piece of data is in physical memory or on disk?  The operating system keeps track of what data has been paged out to disk.  See https://en.wikipedia.org/wiki/Virtual_memory for details on that.