Substracting many objects from a single object
After you've created all the objects you want to "remove" from the wall, Join them. (Control + J in Blender 2.79).  Then you only have two objects, and you can do the Boolean subtraction operation.