Fix the non cython build for cpp extensions
- [ ] closes #19668
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

cc @TomAugspurger @jreback 

This _should_ fix the problem with building with a non Cython environment. The problem is that we assume extensions are C everywhere and I didn't realize that this was a test path.

Also setup.py might be in need of a rethink, there's a lot of special conditions that I think could be cleaned up with some better formatting. 