How many cell membranes are oxygen and carbon dioxide diffuse through in the lungs?
Oxigen goes from the alveolar's lumen to the cytoplasm of the erythrocyte, and that's 5 membranes:


the "top" of the alveolar epithelial cell
the "bottom" of such cell
the "top" of the endothelial cell (capillary)
the "bottom" of such cell
the erythrocyte membrane


You got all the cells right, but your only problem was this: oxygen diffuses through the cell membrane entering the cell, moves through the cytoplasm, and diffuses through the membrane again exiting the cell. So, for each cell, you have to count 2 membranes. For the last one, the erythrocyte, you have only 1 membrane (because it is $\ce{O2}$ final destination). 

For the $\ce{CO2}$ the situation is a little bit more tricky. We have the same 4 membranes (2x epithelial cell and 2x capillary), but $\ce{CO2}$ can come from 2 locations:


from the erythrocyte, where it is formed from $\ce{H2CO3}$ (by the reaction $\ce{H2CO3 -&gt; H2O + CO2}$) or released from the N-terminal group of proteins, like haemoglobin (where it has previously bound)
from the plasma (around 9% of the $\ce{CO2}$).


In the first case we have 5 membranes, and in the second case just 4.

So, the correct answer is D.