How exactly does SPV work? Here is what I have so far:


A mobile connects to a node using rpc. 
The mobile then asks the node, to give him a set of transactions for a set of public keys.
The node, then looks for any transactions that match the public keys given and returns it back to the mobile.


The mobile can setup a bloom filter, so that some of keys included are not his?

Also, when the node receives a request, it needs to scan the blockchain all over again, to get the transaction history?
A mobile connects to a node using rpc.


Nodes don't connect each other using RPC. RPC is for node owners, and usually has a long password.


  The mobile then asks the node, to give him a set of transactions for a set of public keys.


Not exactly. First of all, block headers have a part that's named "Merkle root" that allows clients to check if a transaction was in the block. In theory, as described in the whitepaper, SPV nodes can keep block headers, and ask other nodes for Merkle proof of their transaction. In practice, it's more complicated. Currently 99% of the SPV nodes use BIP 37. Besides, better SPV methods such as BIP 157 and Neutrino are being researched.


  Also, when the node receives a request, it needs to scan the blockchain all over again, to get the transaction history?


Nodes do need to scan the whole blockchain for a transaction if they don't know its block height, unless they use txindex=1, etc... If they do, accessing it is trivial.