Does oil break down condoms?
First off, may I say that I applaud your decision to test this through an experiment. It's rare to see that than I would like.

Now, on to the matter at hand. It's fairly well known from industrial chemistry that non-polar solvents degrade latex quite heavily.

I work with latex seals a lot, and the hexanes we use routinely break the seals down in under a day. Of course, if you're lubricating your condoms with hexanes, you're a) an idiot or b) absolutely insane.

A paper I managed to find suggests that there really isn't too much direct data on condoms, and it muses that the warnings might have arisen from industry, where nonpolar solvents decidedly do degrade latex. 

To find out, they did a burst experiment with condoms that had been treated with various oils. Glycerol and Vaseline-treated condoms showed a very, very minor decrease in strength, while mineral oil/baby oil-treated ones burst at less than 10% of the volume of an untreated condom.

They also found that 10 month-old condoms have half the burst volume of 1-month old ones, so you could argue that using 1-month-old condoms that have been slathered in Vaseline is still much safer than using older ones.

As for the actual chemistry of the weakening, I honestly don't know. If I were to hazard a guess, I would note that the latex looks like a bunch of ethylenes glued together, 

so my guess would be that the solvents get between the chains and force them apart, weakening them. For this to happen, the solvent must be nonpolar, but still small enough to slip between the chains of the polymer.

That's probably why vaseline and canola oil don't have much of an effect---they're just too big to fit between the chains. Again though, I don't know for sure, so don't quote me on this last paragraph.