When an airship climbs the hydrogen in the ship's cells expand. Does this mean that when the airship descends the empty cells fill with air?
First of all, Zeppelins carried tons of water ballast in order to compensate for hydrogen seeping from the gas cells or being vented when the ship had climbed too high. Before launch, the crew calculated the ballast need based on the planned length of the flight and weather conditions. Also, the ship lost mass continuously by burning off fuel. Below you see the calculation sheet of LZ 40 from May 1915. The ship took 7.5 tons of water, distributed over several tanks along its length.

The top row of the sheet lists fuel (Benzin, 9320 kg in total), then oil (520 kg), then a side view of the ship with its 16 gas cells and below that the water ballast. The crew weighed 334 kg.
On the ground the gas bags were never full. In the later WW I Zeppelin models, only one third of the internal volume was taken up by the gas bags and their hydrogen while the rest of the hull was filled with air. As the airship climbed, the hydrogen in its cells expanded and pushed the air out. On the way down the reverse happened. The gas bags were flexible and sufficiently airtight so no air would enter them.
The photo below is from airships.net and shows the LZ-129 Hindenburg under construction. The gas cell is partially filled and is attached to the circular frames over two thirds of their circumference to distribute the lift of a partially filled cell as evenly as possible. Wire bracing distributed the load over the full frame. In the center of the hull you can see a walkway for inspection; the gas cell had a central tunnel to allow lengthwise movement over the full length of the ship. The vertical funnels for venting leaked hydrogen are not yet installed.

This is only possible with rigid ships; blimps use ballonets for the same purpose but have to pressurize them a bit in order to keep their envelope taut.