Zirconium oxygen sensors for car exhaust for use in low temperature environment
I have never tried using one in a plastic container, but the body is made of metal so heat transfers relatively well. After a flame test or testing plugged into an actual exhaust I made sure to not touch the sensor with my bare fingers.

A generic plastic container will almost certainly not be able to deal with the hot operating temperatures, especially if you have to rely at all on the exhaust to heat up the sensor.

Depending on the application, you could try getting exhaust tubing with an O2 sensor port and flow your gas through the tubing past your sensor. The tubing is relatively cheap to get, you may even be able to find something at a local junk yard.