AB testing : When AA testing doesn't work
I 'solved' the issue with Oracle's support, I still post an answer here because some of what I encountered can be useful :  According to Oracle's support, this bug in AB testing programs was known for the last 9 month, but corrected only this week-end.

So my problem is supposedly solved (I will re-launch a test and see if results are now consistent), but I still have some pointers if someone encounters the same problem I did :


Make sure this is not a statistical artifact : test your hypothesis on several independant periods. I used a Fischer test because it is exact for a binomial framework, it is what you should use if your population/conversion rate is too small for a normal approximation
Respect hypothesis testing framework : start your test after you formulated your hypothesis
Get a very very low p-value : No one will listen to your complaint about faulty AB testing if you come up with a 3% p-value, and they would be right. A bayesian analysis probably points to a false positive. 
Make sure the random numbers that are used to split the populations are actually random (and not, say a 50/50 split)
Imagine all the possibilities that your reporting might be wrong. I thought I had imagined them all, but if you read below the real reason behind the bug, you will find that faulty reporting can be hidden very far away in parts of the environment you don't control or even have a view in.


I have the actual answer from Oracle that describes the bug that happened, in case someone has the exact same problem or wants to know how this kind of bug happens : 

For programs triggered by event, their engine does not have the capability to associate the different open and clicks events to the correct email. In the case several emails have been sent through the same program, Responsys assumes the event concerns the earliest email that has been sent through this program.

In my case, the program could have been sent several times during the period.

In the period where I was not splitting my emails through two branches, the name of the campaign I used was already A  (the same than one of my branches in my testing period)

You can see where this is going : Every customer opening any email was marked as having read the "A" email, as long as they had been into the program in the 3 months before I started my AB testing, which caused a faulty reporting.

So actually both branches were sent to the correct customers, and it is just a reporting problem that comes from an approximative algorithm on Responsys side.

The workaround I will try now is simply to create two brand new "campaign" objects before starting a test rather than using a campaign that already existed.