How to make windows into a wall?
Select the face where you want the window,



press E to extrude and move the newly created wall in the direction you want the extrusion.



You can delete then the selected face.



EDIT Another quick way to make a window is using the inset tool.

Select the face on which you want to make the window, press I and drag the mouse. That will create a new face of similar proportions connected to the edges of the original wall. Then extrude by pressing E and dragging the mouse to create the depth for the window.