What is the substance in the new water cooler filter that give bubbles and taste a bit like soap
The most likely cause of a "soapy" taste and sudsing from filtered water is a reverse osmosis filter, for which the usual membrane preservative used to keep the filter from degrading before its first use is plain old glycerin, a common ingredient in and byproduct of traditional soap (though most cleansing products nowadays, even those marketed as soaps, are technically anionic surfactant "detergents", based on alkyl-sulfates). The glycerin is hygroscopic, meaning it will attract and hold water on the membrane surface to keep it from drying out.

An activated carbon filter sometimes has a similar preservative to improve the material's shelf life, or else a binder that allows the manufacturer to mold a block of the carbon rather than loading loose powder into filter housings, which reduces carbon dust in the factory (a major fire and health hazard of these types of plants).

In all cases, the manufacturer usually recommends flushing any new filter with between one and five gallons of source water to wash out these substances, along with any loose filter media or other byproducts of manufacturing. This obviously didn't happen in your case. The glycerin in the water won't kill you; it's an FDA-approved food additive used as a sweetener and "plasticizer" (it's used in cake frosting for instance, to avoid it hardening after it sets). In sufficient doses it's an irritant to the mucous membranes and the GI tract, which can cause nausea and diarrhea, but you'd have to eat almost a kilo of the pure stuff for it to become dangerously toxic.