What sandpaper grit should I use between coats of varnish?
1] Wait. It may take days or weeks for the varnish to harden enough to sand without it "balling up" or being "gummy". This is the most important thing.
2] Test it gently from time to time with 120 or 150 grit until it sands cleanly, no gumminess. Then scuff the whole thing down gently, just knock off the high spots and ridges, avoid breaking through to bare wood. 
3] Lay on another coat, try to brush in one direction, "flow" the varnish on and don't worry about small streaks, varnish is "self levelling" if it's not too cold.
4] Follow the same steps using 220 grit. This should be a decent finish.
5] Depending how far you want to go you can work up in stages to 1200 grit and polishing compound, but that's extreme.
6] When brushing varnish, what you did wasn't entirely wrong. Cross brushing then linear brushing to "work" the varnish onto/into the wood is good but you have to do it fast before the varnish starts to "set up".
Good luck and enjoy.