Revert CI changes from 27542
- [X] closes #27546
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Think this was all a perfect storm of temporary issues, so let's see what reverting yields. 

Note: didn't revert the sudo tag removal on purpose