Cuda error, Out of memory while I have 4GB of GPU
You don't have 4GB available to render by using 2 GPUS with 2GB each: you have only 2GB.

If you look at the specs for the card on the Nvidia Webstite It states clearly that 

The memory configuration is: 4096 MB....(2048 MB per GPU) GDDR5

The memory used multiple GPUs is not added or shared, but each one uses its own pool of vRAM.

The same is true when using multiple cards on the same machine, and on top of that, the memory available to render in cycles is limited by the amount on the smallest of the cards. (For example if you have a 4 GB card and a 2GB one, you would only be able to use 2GB).

Keep in mind also that if any of the cards is plugged to a monitor, or multiple ones, there will be even less vRAM available. All other apps and windows that are open will use some of the resources of your machine.