Extruding, other side does not have face
If you extrude a face connected to other faces, the extrusion will leave a hole.



But if you extrude a disconnected face, you will have a solid shape. If you press Y to disconnect the shape before extruding it, you will make it solid and have that bottom face.

EDIT:
But, beware, if you do this, you will get weird geometry. Things might not be connected the way you want. I feel a little dirty, doing it. :)