Lightning network: Open a channel with a peer that is not a full node? Is it possible to open a channel with a peer that is not a full node? Or to exchange lightning payments between two peers that are not full nodes, they both have to open a channel with full lightning node(s)?

I saw this map
 https://lnmainnet.gaben.win/ and I was wondering if all the graph nodes were full nodes.
Whether you run a full node or a lightweight node is a first layer question. It is a question about how you keep track about what happens on the blockchain. And even though this is important for you, the Lightning nodes that are communicating with your Lightning node do actually not care about it and in fact they don't even (have to) know about your Bitcoin node.

So do not confuse Layer 1 (Bitcoin / Blockchain) with Layer 2 (off-chain payments / Lightning). On Layer 2 there are no "full nodes".

What we do have on Layer 2 however, are channels that you announce to the network and private channels (that you don't announce). 
So if Alice and Bob have a private channel, they can use it to pay each other, but they will not route any third party payments through it.

Yes, the channels that you will see on any Lightning network graph are only announced ones, they private cannot be seen. This is also the reason for why it is difficult to estimate the real size of the Lightning Network.