How can I close a rectangular PVC pipe without using glue?
Cork/Foam/Anything that can be molded to fit the hole snuggly.

Also a they make  caps that extend slightly into the pvc pipe., find a rubber seal (old bike inner tube, large rubber band etc..)attach it to the cap, and then shove it in the hole.