CLN: Remove unnecessary code
`frozenset()` and `frozenset([])` essentially produce identical results, and thus there is no need to pass an empty list to create an empty `frozenset`.

- [ ] tests added / passed (CI is still running...)
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry (no new feature)
