What do I use for backfilling septic pipe leaving my home?
I'd use pea gravel, sand, or flowable fill.  You want to avoid settling which could cause stress on the pipe and possibly break it or cause it to leak.  So fill materials that need little to no compaction are best.
The dirt that you took out will not compact well and it will settle over time.  I'd only use it on top of the fill that covers the top of the pipe.