CI: faster CI builds
#7971 recently changed use to use conda on the 5 main builds

installs now take 300+ seconds, vs 100+ before.

I think a big cause is the installation of some basic libraries that conda installs by default
e.g. Python 2.7.8. But I don't think this is necessary and maybe can skip some of these
