What adhesive or glue should I use to attach velcro to PVC pipe?
I don't have an adhesive suggestion for you, but you could use hose clamps. Put one on the top and one on the bottom of a rectangular section of velcro and clamp it onto the pipe. (Make sure to not over-tighten it.)

Actually, if you could work up a way to attach the fabric to the hose clamps and skip the velcro altogether, that would cut your hose clamp needs by half. They'd be around a buck at the hardware store.