Filling potholes in my dirt driveway
Stick with the crushed rock.  In the deeper holes you could start with 1-1/4"minus.  As you get closer to the surface you go smaller and smaller.  forget the dirt.  you are wasting your time.