What material should I use for the frame of my lighter-than-air craft?
Balsa wood is likely to provide the best strength versus weight ratio of any building material you'll find. Peter Sripol recently posted a YouTube video of his construction, a rigid airship. Pipe cleaners contain a metal spine adding too much weight, in my opinion.
You may want to consider to use thin lines for suspending the gondola. Spectra fishing line is made of the same material as the plastic bags and is notoriously strong. Extending from that line of thought, you could use either balsa struts (hobby shop sourced) or suspend the gondola with strips of plastic from the bags.
Peter attempted to use helium but found it lacked the necessary lifting power. It appears that he used aircraft modeling tissue to keep the weight down, but I suspect your idea of using super thin plastic is a good one.
I have ordinary small trash bags purchased from a big box store. Only yesterday, I measured the thickness of the plastic at 0.0007 inches, 0.0178 mm which may be the thinnest stuff one could find. The bags are relatively large when slit down one side and the bottom, easier to use than a grocery bag, which I now measured at 0.0008 inches thick. At US$4.48 for a box of 80, it's low cost and readily available. If you require a larger contiguous surface area, the same manufacturer offers an 8 gallon size, but I cannot attest to the thickness of that product. There are no specifications for thickness in the product listing. My figures come from direct measurement with a micrometer capable of reading to one ten-thousandth of an inch.

Home Depot lists 0.7 mil (0.0007 inches) drop cloth but I know that the material is not as durable as the trash bags from WM.
Peter Sripol eventually used hydrogen to get the necessary buoyancy for his vessel and typical of Peter, he ignited the craft during its (obviously) last flight. Hydrogen does not explode, but it will burn as it gains access to oxygen. An explosion would require a suitable mix of oxygen and hydrogen, but your dirigible would/should be inflated with only the one component.
EDIT: I did a quick search for black plastic drop cloth and found I was wrong about the thickness. There's some four mill stuff out there. I suggest black if you don't have an objection, as you'll get solar heating benefit as well. There are also those solar balloons that work on the same principle. Inflate, expose to sunlight, watch them float away!