Clean up ufuncs post numpy bump
Found an old branch lying around I had forgotten about after the numpy bump, which had been requested by @TomAugspurger in [this comment](https://github.com/pandas-dev/pandas/pull/25554#pullrequestreview-212474594)
