Paver installation - excavation limitation
I have run into this many times usually with light duty drainage pipe for the gutters . I really like a solid base I use more rock and less sand but 6” for a patio is good. I would expose a little of the pipe and stop at 5”. 

When packing the gravel work to the pipe from both sides  but don’t pack directly over it until both sides are packed. This reduces breakage of the pipe if it is light duty drainage pipe like I have run into. I also use water while packing this helps to pack the rock into the dirt and work the fines from packing into the cracks.

I use just enough sand to level everything in reality a deep spot might be 1/2” for me with a thin spot being close to 1/4”. I get it flat with a 3’ wide rake that has teeth on 1 side and smooth edge on the other side, by working at 90 degree angles he sand flattens out quickly. By over lapping each pass with 2/2 of the rake then turning and working at 90 doing the same the low spots show up quickly after I get the sand very close I use a mist nozzle to wet it down and a last pass, wet sand is harder to work that’s why I get flat prior to wetting it down, a mist nozzle takes longer but won’t dig ruts in the sand. 

A border around the area is best but is really needed if more sand is used when it rains water pulls the sand out at the edges of there is no border. A thin layer of sand won’t have the sand migration as bad even with small pavers. Last make sure to have a slight grade away from the house, yes the rock will help to create a drain field but a well packed base and poly sand will hold water quite well so having a slight grade away from the house is best.. 
I have put in many patios with pavers if crushed quarry or shale is used the rock locks together and can last decades don’t use crushed river it doesn’t lock together I have learned this the hard way living in Oregon where crushed river is much cheaper , I only use that for French drains where o don’t want it to pack together. Hope this helps , 4” of rock will work 5 is better but 4-1/2 with 1/2 of sand will reduce your chances of breaking the pipe and still provide a good base for your pavers.